Limiting Distributions of Functionals of Markov Chains.

DTIC Science & Technology

1984-08-01

limiting distributions; periodic * nonhomoger.,!ous Poisson processes . 19 ANS? MACY IConuui oe nonoe’ee if necorglooy and edern thty by block numbers...homogeneous Poisson processes is of interest in itself. The problem considered in this paper is of interest in the theory of partially observable...where we obtain the limiting distribution of the interevent times. Key Words: Markov Chains, Limiting Distributions, Periodic Nonhomogeneous Poisson

Sensitivity Study for Long Term Reliability

NASA Technical Reports Server (NTRS)

White, Allan L.

2008-01-01

This paper illustrates using Markov models to establish system and maintenance requirements for small electronic controllers where the goal is a high probability of continuous service for a long period of time. The system and maintenance items considered are quality of components, various degrees of simple redundancy, redundancy with reconfiguration, diagnostic levels, periodic maintenance, and preventive maintenance. Markov models permit a quantitative investigation with comparison and contrast. An element of special interest is the use of conditional probability to study the combination of imperfect diagnostics and periodic maintenance.

How Complex, Probable, and Predictable is Genetically Driven Red Queen Chaos?

PubMed

Duarte, Jorge; Rodrigues, Carla; Januário, Cristina; Martins, Nuno; Sardanyés, Josep

2015-12-01

Coevolution between two antagonistic species has been widely studied theoretically for both ecologically- and genetically-driven Red Queen dynamics. A typical outcome of these systems is an oscillatory behavior causing an endless series of one species adaptation and others counter-adaptation. More recently, a mathematical model combining a three-species food chain system with an adaptive dynamics approach revealed genetically driven chaotic Red Queen coevolution. In the present article, we analyze this mathematical model mainly focusing on the impact of species rates of evolution (mutation rates) in the dynamics. Firstly, we analytically proof the boundedness of the trajectories of the chaotic attractor. The complexity of the coupling between the dynamical variables is quantified using observability indices. By using symbolic dynamics theory, we quantify the complexity of genetically driven Red Queen chaos computing the topological entropy of existing one-dimensional iterated maps using Markov partitions. Co-dimensional two bifurcation diagrams are also built from the period ordering of the orbits of the maps. Then, we study the predictability of the Red Queen chaos, found in narrow regions of mutation rates. To extend the previous analyses, we also computed the likeliness of finding chaos in a given region of the parameter space varying other model parameters simultaneously. Such analyses allowed us to compute a mean predictability measure for the system in the explored region of the parameter space. We found that genetically driven Red Queen chaos, although being restricted to small regions of the analyzed parameter space, might be highly unpredictable.

Open Markov Processes and Reaction Networks

NASA Astrophysics Data System (ADS)

Swistock Pollard, Blake Stephen

We begin by defining the concept of `open' Markov processes, which are continuous-time Markov chains where probability can flow in and out through certain `boundary' states. We study open Markov processes which in the absence of such boundary flows admit equilibrium states satisfying detailed balance, meaning that the net flow of probability vanishes between all pairs of states. External couplings which fix the probabilities of boundary states can maintain such systems in non-equilibrium steady states in which non-zero probability currents flow. We show that these non-equilibrium steady states minimize a quadratic form which we call 'dissipation.' This is closely related to Prigogine's principle of minimum entropy production. We bound the rate of change of the entropy of a driven non-equilibrium steady state relative to the underlying equilibrium state in terms of the flow of probability through the boundary of the process. We then consider open Markov processes as morphisms in a symmetric monoidal category by splitting up their boundary states into certain sets of `inputs' and `outputs.' Composition corresponds to gluing the outputs of one such open Markov process onto the inputs of another so that the probability flowing out of the first process is equal to the probability flowing into the second. Tensoring in this category corresponds to placing two such systems side by side. We construct a `black-box' functor characterizing the behavior of an open Markov process in terms of the space of possible steady state probabilities and probability currents along the boundary. The fact that this is a functor means that the behavior of a composite open Markov process can be computed by composing the behaviors of the open Markov processes from which it is composed. We prove a similar black-boxing theorem for reaction networks whose dynamics are given by the non-linear rate equation. Along the way we describe a more general category of open dynamical systems where composition corresponds to gluing together open dynamical systems.

[Development of Markov models for economics evaluation of strategies on hepatitis B vaccination and population-based antiviral treatment in China].

PubMed

Yang, P C; Zhang, S X; Sun, P P; Cai, Y L; Lin, Y; Zou, Y H

2017-07-10

Objective: To construct the Markov models to reflect the reality of prevention and treatment interventions against hepatitis B virus (HBV) infection, simulate the natural history of HBV infection in different age groups and provide evidence for the economics evaluations of hepatitis B vaccination and population-based antiviral treatment in China. Methods: According to the theory and techniques of Markov chain, the Markov models of Chinese HBV epidemic were developed based on the national data and related literature both at home and abroad, including the settings of Markov model states, allowable transitions and initial and transition probabilities. The model construction, operation and verification were conducted by using software TreeAge Pro 2015. Results: Several types of Markov models were constructed to describe the disease progression of HBV infection in neonatal period, perinatal period or adulthood, the progression of chronic hepatitis B after antiviral therapy, hepatitis B prevention and control in adults, chronic hepatitis B antiviral treatment and the natural progression of chronic hepatitis B in general population. The model for the newborn was fundamental which included ten states, i.e . susceptiblity to HBV, HBsAg clearance, immune tolerance, immune clearance, low replication, HBeAg negative CHB, compensated cirrhosis, decompensated cirrhosis, hepatocellular carcinoma (HCC) and death. The susceptible state to HBV was excluded in the perinatal period model, and the immune tolerance state was excluded in the adulthood model. The model for general population only included two states, survive and death. Among the 5 types of models, there were 9 initial states assigned with initial probabilities, and 27 states for transition probabilities. The results of model verifications showed that the probability curves were basically consistent with the situation of HBV epidemic in China. Conclusion: The Markov models developed can be used in economics evaluation of hepatitis B vaccination and treatment for the elimination of HBV infection in China though the structures and parameters in the model have uncertainty with dynamic natures.

Activation rates for nonlinear stochastic flows driven by non-Gaussian noise

NASA Astrophysics Data System (ADS)

van den Broeck, C.; Hänggi, P.

1984-11-01

Activation rates are calculated for stochastic bistable flows driven by asymmetric dichotomic Markov noise (a two-state Markov process). This noise contains as limits both a particular type of non-Gaussian white shot noise and white Gaussian noise. Apart from investigating the role of colored noise on the escape rates, one can thus also study the influence of the non-Gaussian nature of the noise on these rates. The rate for white shot noise differs in leading order (Arrhenius factor) from the corresponding rate for white Gaussian noise of equal strength. In evaluating the rates we demonstrate the advantage of using transport theory over a mean first-passage time approach for cases with generally non-white and non-Gaussian noise sources. For white shot noise with exponentially distributed weights we succeed in evaluating the mean first-passage time of the corresponding integro-differential master-equation dynamics. The rate is shown to coincide in the weak noise limit with the inverse mean first-passage time.

A mathematical approach for evaluating Markov models in continuous time without discrete-event simulation.

PubMed

van Rosmalen, Joost; Toy, Mehlika; O'Mahony, James F

2013-08-01

Markov models are a simple and powerful tool for analyzing the health and economic effects of health care interventions. These models are usually evaluated in discrete time using cohort analysis. The use of discrete time assumes that changes in health states occur only at the end of a cycle period. Discrete-time Markov models only approximate the process of disease progression, as clinical events typically occur in continuous time. The approximation can yield biased cost-effectiveness estimates for Markov models with long cycle periods and if no half-cycle correction is made. The purpose of this article is to present an overview of methods for evaluating Markov models in continuous time. These methods use mathematical results from stochastic process theory and control theory. The methods are illustrated using an applied example on the cost-effectiveness of antiviral therapy for chronic hepatitis B. The main result is a mathematical solution for the expected time spent in each state in a continuous-time Markov model. It is shown how this solution can account for age-dependent transition rates and discounting of costs and health effects, and how the concept of tunnel states can be used to account for transition rates that depend on the time spent in a state. The applied example shows that the continuous-time model yields more accurate results than the discrete-time model but does not require much computation time and is easily implemented. In conclusion, continuous-time Markov models are a feasible alternative to cohort analysis and can offer several theoretical and practical advantages.

On the Stability of Jump-Linear Systems Driven by Finite-State Machines with Markovian Inputs

NASA Technical Reports Server (NTRS)

Patilkulkarni, Sudarshan; Herencia-Zapana, Heber; Gray, W. Steven; Gonzalez, Oscar R.

2004-01-01

This paper presents two mean-square stability tests for a jump-linear system driven by a finite-state machine with a first-order Markovian input process. The first test is based on conventional Markov jump-linear theory and avoids the use of any higher-order statistics. The second test is developed directly using the higher-order statistics of the machine s output process. The two approaches are illustrated with a simple model for a recoverable computer control system.

Net Surface Flux Budget Over Tropical Oceans Estimated from the Tropical Rainfall Measuring Mission (TRMM)

NASA Astrophysics Data System (ADS)

Fan, Tai-Fang

We begin by defining the concept of `open' Markov processes, which are continuous-time Markov chains where probability can flow in and out through certain `boundary' states. We study open Markov processes which in the absence of such boundary flows admit equilibrium states satisfying detailed balance, meaning that the net flow of probability vanishes between all pairs of states. External couplings which fix the probabilities of boundary states can maintain such systems in non-equilibrium steady states in which non-zero probability currents flow. We show that these non-equilibrium steady states minimize a quadratic form which we call 'dissipation.' This is closely related to Prigogine's principle of minimum entropy production. We bound the rate of change of the entropy of a driven non-equilibrium steady state relative to the underlying equilibrium state in terms of the flow of probability through the boundary of the process. We then consider open Markov processes as morphisms in a symmetric monoidal category by splitting up their boundary states into certain sets of `inputs' and `outputs.' Composition corresponds to gluing the outputs of one such open Markov process onto the inputs of another so that the probability flowing out of the first process is equal to the probability flowing into the second. Tensoring in this category corresponds to placing two such systems side by side. We construct a `black-box' functor characterizing the behavior of an open Markov process in terms of the space of possible steady state probabilities and probability currents along the boundary. The fact that this is a functor means that the behavior of a composite open Markov process can be computed by composing the behaviors of the open Markov processes from which it is composed. We prove a similar black-boxing theorem for reaction networks whose dynamics are given by the non-linear rate equation. Along the way we describe a more general category of open dynamical systems where composition corresponds to gluing together open dynamical systems.

Magneto - Optical Imaging of Superconducting MgB2 Thin Films

NASA Astrophysics Data System (ADS)

Hummert, Stephanie Maria

We begin by defining the concept of `open' Markov processes, which are continuous-time Markov chains where probability can flow in and out through certain `boundary' states. We study open Markov processes which in the absence of such boundary flows admit equilibrium states satisfying detailed balance, meaning that the net flow of probability vanishes between all pairs of states. External couplings which fix the probabilities of boundary states can maintain such systems in non-equilibrium steady states in which non-zero probability currents flow. We show that these non-equilibrium steady states minimize a quadratic form which we call 'dissipation.' This is closely related to Prigogine's principle of minimum entropy production. We bound the rate of change of the entropy of a driven non-equilibrium steady state relative to the underlying equilibrium state in terms of the flow of probability through the boundary of the process. We then consider open Markov processes as morphisms in a symmetric monoidal category by splitting up their boundary states into certain sets of `inputs' and `outputs.' Composition corresponds to gluing the outputs of one such open Markov process onto the inputs of another so that the probability flowing out of the first process is equal to the probability flowing into the second. Tensoring in this category corresponds to placing two such systems side by side. We construct a `black-box' functor characterizing the behavior of an open Markov process in terms of the space of possible steady state probabilities and probability currents along the boundary. The fact that this is a functor means that the behavior of a composite open Markov process can be computed by composing the behaviors of the open Markov processes from which it is composed. We prove a similar black-boxing theorem for reaction networks whose dynamics are given by the non-linear rate equation. Along the way we describe a more general category of open dynamical systems where composition corresponds to gluing together open dynamical systems.

Boron Carbide Filled Neutron Shielding Textile Polymers

NASA Astrophysics Data System (ADS)

Manzlak, Derrick Anthony

We begin by defining the concept of `open' Markov processes, which are continuous-time Markov chains where probability can flow in and out through certain `boundary' states. We study open Markov processes which in the absence of such boundary flows admit equilibrium states satisfying detailed balance, meaning that the net flow of probability vanishes between all pairs of states. External couplings which fix the probabilities of boundary states can maintain such systems in non-equilibrium steady states in which non-zero probability currents flow. We show that these non-equilibrium steady states minimize a quadratic form which we call 'dissipation.' This is closely related to Prigogine's principle of minimum entropy production. We bound the rate of change of the entropy of a driven non-equilibrium steady state relative to the underlying equilibrium state in terms of the flow of probability through the boundary of the process. We then consider open Markov processes as morphisms in a symmetric monoidal category by splitting up their boundary states into certain sets of `inputs' and `outputs.' Composition corresponds to gluing the outputs of one such open Markov process onto the inputs of another so that the probability flowing out of the first process is equal to the probability flowing into the second. Tensoring in this category corresponds to placing two such systems side by side. We construct a `black-box' functor characterizing the behavior of an open Markov process in terms of the space of possible steady state probabilities and probability currents along the boundary. The fact that this is a functor means that the behavior of a composite open Markov process can be computed by composing the behaviors of the open Markov processes from which it is composed. We prove a similar black-boxing theorem for reaction networks whose dynamics are given by the non-linear rate equation. Along the way we describe a more general category of open dynamical systems where composition corresponds to gluing together open dynamical systems.

Parallel Unstructured Grid Generation for Complex Real-World Aerodynamic Simulations

NASA Astrophysics Data System (ADS)

Zagaris, George

We begin by defining the concept of `open' Markov processes, which are continuous-time Markov chains where probability can flow in and out through certain `boundary' states. We study open Markov processes which in the absence of such boundary flows admit equilibrium states satisfying detailed balance, meaning that the net flow of probability vanishes between all pairs of states. External couplings which fix the probabilities of boundary states can maintain such systems in non-equilibrium steady states in which non-zero probability currents flow. We show that these non-equilibrium steady states minimize a quadratic form which we call 'dissipation.' This is closely related to Prigogine's principle of minimum entropy production. We bound the rate of change of the entropy of a driven non-equilibrium steady state relative to the underlying equilibrium state in terms of the flow of probability through the boundary of the process. We then consider open Markov processes as morphisms in a symmetric monoidal category by splitting up their boundary states into certain sets of `inputs' and `outputs.' Composition corresponds to gluing the outputs of one such open Markov process onto the inputs of another so that the probability flowing out of the first process is equal to the probability flowing into the second. Tensoring in this category corresponds to placing two such systems side by side. We construct a `black-box' functor characterizing the behavior of an open Markov process in terms of the space of possible steady state probabilities and probability currents along the boundary. The fact that this is a functor means that the behavior of a composite open Markov process can be computed by composing the behaviors of the open Markov processes from which it is composed. We prove a similar black-boxing theorem for reaction networks whose dynamics are given by the non-linear rate equation. Along the way we describe a more general category of open dynamical systems where composition corresponds to gluing together open dynamical systems.

Polymeric Radiation Shielding for Applications in Space: Polyimide Synthesis and Modeling of Multi-Layered Polymeric Shields

NASA Astrophysics Data System (ADS)

Schiavone, Clinton Cleveland

We begin by defining the concept of `open' Markov processes, which are continuous-time Markov chains where probability can flow in and out through certain `boundary' states. We study open Markov processes which in the absence of such boundary flows admit equilibrium states satisfying detailed balance, meaning that the net flow of probability vanishes between all pairs of states. External couplings which fix the probabilities of boundary states can maintain such systems in non-equilibrium steady states in which non-zero probability currents flow. We show that these non-equilibrium steady states minimize a quadratic form which we call 'dissipation.' This is closely related to Prigogine's principle of minimum entropy production. We bound the rate of change of the entropy of a driven non-equilibrium steady state relative to the underlying equilibrium state in terms of the flow of probability through the boundary of the process. We then consider open Markov processes as morphisms in a symmetric monoidal category by splitting up their boundary states into certain sets of `inputs' and `outputs.' Composition corresponds to gluing the outputs of one such open Markov process onto the inputs of another so that the probability flowing out of the first process is equal to the probability flowing into the second. Tensoring in this category corresponds to placing two such systems side by side. We construct a `black-box' functor characterizing the behavior of an open Markov process in terms of the space of possible steady state probabilities and probability currents along the boundary. The fact that this is a functor means that the behavior of a composite open Markov process can be computed by composing the behaviors of the open Markov processes from which it is composed. We prove a similar black-boxing theorem for reaction networks whose dynamics are given by the non-linear rate equation. Along the way we describe a more general category of open dynamical systems where composition corresponds to gluing together open dynamical systems.

Processing and Conversion of Algae to Bioethanol

NASA Astrophysics Data System (ADS)

Kampfe, Sara Katherine

We begin by defining the concept of `open' Markov processes, which are continuous-time Markov chains where probability can flow in and out through certain `boundary' states. We study open Markov processes which in the absence of such boundary flows admit equilibrium states satisfying detailed balance, meaning that the net flow of probability vanishes between all pairs of states. External couplings which fix the probabilities of boundary states can maintain such systems in non-equilibrium steady states in which non-zero probability currents flow. We show that these non-equilibrium steady states minimize a quadratic form which we call 'dissipation.' This is closely related to Prigogine's principle of minimum entropy production. We bound the rate of change of the entropy of a driven non-equilibrium steady state relative to the underlying equilibrium state in terms of the flow of probability through the boundary of the process. We then consider open Markov processes as morphisms in a symmetric monoidal category by splitting up their boundary states into certain sets of `inputs' and `outputs.' Composition corresponds to gluing the outputs of one such open Markov process onto the inputs of another so that the probability flowing out of the first process is equal to the probability flowing into the second. Tensoring in this category corresponds to placing two such systems side by side. We construct a `black-box' functor characterizing the behavior of an open Markov process in terms of the space of possible steady state probabilities and probability currents along the boundary. The fact that this is a functor means that the behavior of a composite open Markov process can be computed by composing the behaviors of the open Markov processes from which it is composed. We prove a similar black-boxing theorem for reaction networks whose dynamics are given by the non-linear rate equation. Along the way we describe a more general category of open dynamical systems where composition corresponds to gluing together open dynamical systems.

The Development of the CALIPSO LiDAR Simulator

NASA Astrophysics Data System (ADS)

Powell, Kathleen A.

We begin by defining the concept of `open' Markov processes, which are continuous-time Markov chains where probability can flow in and out through certain `boundary' states. We study open Markov processes which in the absence of such boundary flows admit equilibrium states satisfying detailed balance, meaning that the net flow of probability vanishes between all pairs of states. External couplings which fix the probabilities of boundary states can maintain such systems in non-equilibrium steady states in which non-zero probability currents flow. We show that these non-equilibrium steady states minimize a quadratic form which we call 'dissipation.' This is closely related to Prigogine's principle of minimum entropy production. We bound the rate of change of the entropy of a driven non-equilibrium steady state relative to the underlying equilibrium state in terms of the flow of probability through the boundary of the process. We then consider open Markov processes as morphisms in a symmetric monoidal category by splitting up their boundary states into certain sets of `inputs' and `outputs.' Composition corresponds to gluing the outputs of one such open Markov process onto the inputs of another so that the probability flowing out of the first process is equal to the probability flowing into the second. Tensoring in this category corresponds to placing two such systems side by side. We construct a `black-box' functor characterizing the behavior of an open Markov process in terms of the space of possible steady state probabilities and probability currents along the boundary. The fact that this is a functor means that the behavior of a composite open Markov process can be computed by composing the behaviors of the open Markov processes from which it is composed. We prove a similar black-boxing theorem for reaction networks whose dynamics are given by the non-linear rate equation. Along the way we describe a more general category of open dynamical systems where composition corresponds to gluing together open dynamical systems.

Exploring a Novel Approach to Technical Nuclear Forensics Utilizing Atomic Force Microscopy

NASA Astrophysics Data System (ADS)

Peeke, Richard Scot

We begin by defining the concept of `open' Markov processes, which are continuous-time Markov chains where probability can flow in and out through certain `boundary' states. We study open Markov processes which in the absence of such boundary flows admit equilibrium states satisfying detailed balance, meaning that the net flow of probability vanishes between all pairs of states. External couplings which fix the probabilities of boundary states can maintain such systems in non-equilibrium steady states in which non-zero probability currents flow. We show that these non-equilibrium steady states minimize a quadratic form which we call 'dissipation.' This is closely related to Prigogine's principle of minimum entropy production. We bound the rate of change of the entropy of a driven non-equilibrium steady state relative to the underlying equilibrium state in terms of the flow of probability through the boundary of the process. We then consider open Markov processes as morphisms in a symmetric monoidal category by splitting up their boundary states into certain sets of `inputs' and `outputs.' Composition corresponds to gluing the outputs of one such open Markov process onto the inputs of another so that the probability flowing out of the first process is equal to the probability flowing into the second. Tensoring in this category corresponds to placing two such systems side by side. We construct a `black-box' functor characterizing the behavior of an open Markov process in terms of the space of possible steady state probabilities and probability currents along the boundary. The fact that this is a functor means that the behavior of a composite open Markov process can be computed by composing the behaviors of the open Markov processes from which it is composed. We prove a similar black-boxing theorem for reaction networks whose dynamics are given by the non-linear rate equation. Along the way we describe a more general category of open dynamical systems where composition corresponds to gluing together open dynamical systems.

Modeling of Critically-Stratified Gravity Flows: Application to the Eel River Continental Shelf, Northern California

NASA Astrophysics Data System (ADS)

Scully, Malcolm E.

We begin by defining the concept of `open' Markov processes, which are continuous-time Markov chains where probability can flow in and out through certain `boundary' states. We study open Markov processes which in the absence of such boundary flows admit equilibrium states satisfying detailed balance, meaning that the net flow of probability vanishes between all pairs of states. External couplings which fix the probabilities of boundary states can maintain such systems in non-equilibrium steady states in which non-zero probability currents flow. We show that these non-equilibrium steady states minimize a quadratic form which we call 'dissipation.' This is closely related to Prigogine's principle of minimum entropy production. We bound the rate of change of the entropy of a driven non-equilibrium steady state relative to the underlying equilibrium state in terms of the flow of probability through the boundary of the process. We then consider open Markov processes as morphisms in a symmetric monoidal category by splitting up their boundary states into certain sets of `inputs' and `outputs.' Composition corresponds to gluing the outputs of one such open Markov process onto the inputs of another so that the probability flowing out of the first process is equal to the probability flowing into the second. Tensoring in this category corresponds to placing two such systems side by side. We construct a `black-box' functor characterizing the behavior of an open Markov process in terms of the space of possible steady state probabilities and probability currents along the boundary. The fact that this is a functor means that the behavior of a composite open Markov process can be computed by composing the behaviors of the open Markov processes from which it is composed. We prove a similar black-boxing theorem for reaction networks whose dynamics are given by the non-linear rate equation. Along the way we describe a more general category of open dynamical systems where composition corresponds to gluing together open dynamical systems.

Production of Cyclohexylene-Containing Diamines in Pursuit of Novel Radiation Shielding Materials

NASA Astrophysics Data System (ADS)

Bate, Norah G.

We begin by defining the concept of `open' Markov processes, which are continuous-time Markov chains where probability can flow in and out through certain `boundary' states. We study open Markov processes which in the absence of such boundary flows admit equilibrium states satisfying detailed balance, meaning that the net flow of probability vanishes between all pairs of states. External couplings which fix the probabilities of boundary states can maintain such systems in non-equilibrium steady states in which non-zero probability currents flow. We show that these non-equilibrium steady states minimize a quadratic form which we call 'dissipation.' This is closely related to Prigogine's principle of minimum entropy production. We bound the rate of change of the entropy of a driven non-equilibrium steady state relative to the underlying equilibrium state in terms of the flow of probability through the boundary of the process. We then consider open Markov processes as morphisms in a symmetric monoidal category by splitting up their boundary states into certain sets of `inputs' and `outputs.' Composition corresponds to gluing the outputs of one such open Markov process onto the inputs of another so that the probability flowing out of the first process is equal to the probability flowing into the second. Tensoring in this category corresponds to placing two such systems side by side. We construct a `black-box' functor characterizing the behavior of an open Markov process in terms of the space of possible steady state probabilities and probability currents along the boundary. The fact that this is a functor means that the behavior of a composite open Markov process can be computed by composing the behaviors of the open Markov processes from which it is composed. We prove a similar black-boxing theorem for reaction networks whose dynamics are given by the non-linear rate equation. Along the way we describe a more general category of open dynamical systems where composition corresponds to gluing together open dynamical systems.

Development of Boron-Containing Polyimide Materials and Poly(arylene Ether)s for Radiation Shielding

NASA Astrophysics Data System (ADS)

Collins, Brittani May

We begin by defining the concept of `open' Markov processes, which are continuous-time Markov chains where probability can flow in and out through certain `boundary' states. We study open Markov processes which in the absence of such boundary flows admit equilibrium states satisfying detailed balance, meaning that the net flow of probability vanishes between all pairs of states. External couplings which fix the probabilities of boundary states can maintain such systems in non-equilibrium steady states in which non-zero probability currents flow. We show that these non-equilibrium steady states minimize a quadratic form which we call 'dissipation.' This is closely related to Prigogine's principle of minimum entropy production. We bound the rate of change of the entropy of a driven non-equilibrium steady state relative to the underlying equilibrium state in terms of the flow of probability through the boundary of the process. We then consider open Markov processes as morphisms in a symmetric monoidal category by splitting up their boundary states into certain sets of `inputs' and `outputs.' Composition corresponds to gluing the outputs of one such open Markov process onto the inputs of another so that the probability flowing out of the first process is equal to the probability flowing into the second. Tensoring in this category corresponds to placing two such systems side by side. We construct a `black-box' functor characterizing the behavior of an open Markov process in terms of the space of possible steady state probabilities and probability currents along the boundary. The fact that this is a functor means that the behavior of a composite open Markov process can be computed by composing the behaviors of the open Markov processes from which it is composed. We prove a similar black-boxing theorem for reaction networks whose dynamics are given by the non-linear rate equation. Along the way we describe a more general category of open dynamical systems where composition corresponds to gluing together open dynamical systems.

Magnetization Dynamics and Anisotropy in Ferromagnetic/Antiferromagnetic Ni/NiO Bilayers

NASA Astrophysics Data System (ADS)

Petersen, Andreas

We begin by defining the concept of `open' Markov processes, which are continuous-time Markov chains where probability can flow in and out through certain `boundary' states. We study open Markov processes which in the absence of such boundary flows admit equilibrium states satisfying detailed balance, meaning that the net flow of probability vanishes between all pairs of states. External couplings which fix the probabilities of boundary states can maintain such systems in non-equilibrium steady states in which non-zero probability currents flow. We show that these non-equilibrium steady states minimize a quadratic form which we call 'dissipation.' This is closely related to Prigogine's principle of minimum entropy production. We bound the rate of change of the entropy of a driven non-equilibrium steady state relative to the underlying equilibrium state in terms of the flow of probability through the boundary of the process. We then consider open Markov processes as morphisms in a symmetric monoidal category by splitting up their boundary states into certain sets of `inputs' and `outputs.' Composition corresponds to gluing the outputs of one such open Markov process onto the inputs of another so that the probability flowing out of the first process is equal to the probability flowing into the second. Tensoring in this category corresponds to placing two such systems side by side. We construct a `black-box' functor characterizing the behavior of an open Markov process in terms of the space of possible steady state probabilities and probability currents along the boundary. The fact that this is a functor means that the behavior of a composite open Markov process can be computed by composing the behaviors of the open Markov processes from which it is composed. We prove a similar black-boxing theorem for reaction networks whose dynamics are given by the non-linear rate equation. Along the way we describe a more general category of open dynamical systems where composition corresponds to gluing together open dynamical systems.

Markov switching multinomial logit model: An application to accident-injury severities.

PubMed

Malyshkina, Nataliya V; Mannering, Fred L

2009-07-01

In this study, two-state Markov switching multinomial logit models are proposed for statistical modeling of accident-injury severities. These models assume Markov switching over time between two unobserved states of roadway safety as a means of accounting for potential unobserved heterogeneity. The states are distinct in the sense that in different states accident-severity outcomes are generated by separate multinomial logit processes. To demonstrate the applicability of the approach, two-state Markov switching multinomial logit models are estimated for severity outcomes of accidents occurring on Indiana roads over a four-year time period. Bayesian inference methods and Markov Chain Monte Carlo (MCMC) simulations are used for model estimation. The estimated Markov switching models result in a superior statistical fit relative to the standard (single-state) multinomial logit models for a number of roadway classes and accident types. It is found that the more frequent state of roadway safety is correlated with better weather conditions and that the less frequent state is correlated with adverse weather conditions.

Performance Analysis of Power Saving Class of Type 1 with Both Downlink and Uplink Traffics in IEEE 802.16e

NASA Astrophysics Data System (ADS)

Baek, Sangkyu; Choi, Bong Dae

We investigate power consumption of a mobile station with the power saving class of type 1 in the IEEE 802.16e. We deal with stochastic behavior of mobile station during not only sleep mode period but also awake mode period with both downlink and uplink traffics. Our methods for investigating the power saving class of type 1 are to construct the embedded Markov chain and the semi-Markov chain generated by the embedded Markov chain. To see the effect of the sleep mode, we obtain the average power consumption of a mobile station and the mean queueing delay of a message. Numerical results show that the larger size of the sleep window makes the power consumption of a mobile station smaller and the queueing delay of a downlink message longer.

Continuous-Time Semi-Markov Models in Health Economic Decision Making: An Illustrative Example in Heart Failure Disease Management.

PubMed

Cao, Qi; Buskens, Erik; Feenstra, Talitha; Jaarsma, Tiny; Hillege, Hans; Postmus, Douwe

2016-01-01

Continuous-time state transition models may end up having large unwieldy structures when trying to represent all relevant stages of clinical disease processes by means of a standard Markov model. In such situations, a more parsimonious, and therefore easier-to-grasp, model of a patient's disease progression can often be obtained by assuming that the future state transitions do not depend only on the present state (Markov assumption) but also on the past through time since entry in the present state. Despite that these so-called semi-Markov models are still relatively straightforward to specify and implement, they are not yet routinely applied in health economic evaluation to assess the cost-effectiveness of alternative interventions. To facilitate a better understanding of this type of model among applied health economic analysts, the first part of this article provides a detailed discussion of what the semi-Markov model entails and how such models can be specified in an intuitive way by adopting an approach called vertical modeling. In the second part of the article, we use this approach to construct a semi-Markov model for assessing the long-term cost-effectiveness of 3 disease management programs for heart failure. Compared with a standard Markov model with the same disease states, our proposed semi-Markov model fitted the observed data much better. When subsequently extrapolating beyond the clinical trial period, these relatively large differences in goodness-of-fit translated into almost a doubling in mean total cost and a 60-d decrease in mean survival time when using the Markov model instead of the semi-Markov model. For the disease process considered in our case study, the semi-Markov model thus provided a sensible balance between model parsimoniousness and computational complexity. © The Author(s) 2015.

Improving the Fitness of High-Dimensional Biomechanical Models via Data-Driven Stochastic Exploration

PubMed Central

Bustamante, Carlos D.; Valero-Cuevas, Francisco J.

2010-01-01

The field of complex biomechanical modeling has begun to rely on Monte Carlo techniques to investigate the effects of parameter variability and measurement uncertainty on model outputs, search for optimal parameter combinations, and define model limitations. However, advanced stochastic methods to perform data-driven explorations, such as Markov chain Monte Carlo (MCMC), become necessary as the number of model parameters increases. Here, we demonstrate the feasibility and, what to our knowledge is, the first use of an MCMC approach to improve the fitness of realistically large biomechanical models. We used a Metropolis–Hastings algorithm to search increasingly complex parameter landscapes (3, 8, 24, and 36 dimensions) to uncover underlying distributions of anatomical parameters of a “truth model” of the human thumb on the basis of simulated kinematic data (thumbnail location, orientation, and linear and angular velocities) polluted by zero-mean, uncorrelated multivariate Gaussian “measurement noise.” Driven by these data, ten Markov chains searched each model parameter space for the subspace that best fit the data (posterior distribution). As expected, the convergence time increased, more local minima were found, and marginal distributions broadened as the parameter space complexity increased. In the 36-D scenario, some chains found local minima but the majority of chains converged to the true posterior distribution (confirmed using a cross-validation dataset), thus demonstrating the feasibility and utility of these methods for realistically large biomechanical problems. PMID:19272906

High-Performance Nanocomposites Designed for Radiation Shielding in Space and an Application of GIS for Analyzing Nanopowder Dispersion in Polymer Matrixes

NASA Astrophysics Data System (ADS)

Auslander, Joseph Simcha

We begin by defining the concept of `open' Markov processes, which are continuous-time Markov chains where probability can flow in and out through certain `boundary' states. We study open Markov processes which in the absence of such boundary flows admit equilibrium states satisfying detailed balance, meaning that the net flow of probability vanishes between all pairs of states. External couplings which fix the probabilities of boundary states can maintain such systems in non-equilibrium steady states in which non-zero probability currents flow. We show that these non-equilibrium steady states minimize a quadratic form which we call 'dissipation.' This is closely related to Prigogine's principle of minimum entropy production. We bound the rate of change of the entropy of a driven non-equilibrium steady state relative to the underlying equilibrium state in terms of the flow of probability through the boundary of the process. We then consider open Markov processes as morphisms in a symmetric monoidal category by splitting up their boundary states into certain sets of `inputs' and `outputs.' Composition corresponds to gluing the outputs of one such open Markov process onto the inputs of another so that the probability flowing out of the first process is equal to the probability flowing into the second. Tensoring in this category corresponds to placing two such systems side by side. We construct a `black-box' functor characterizing the behavior of an open Markov process in terms of the space of possible steady state probabilities and probability currents along the boundary. The fact that this is a functor means that the behavior of a composite open Markov process can be computed by composing the behaviors of the open Markov processes from which it is composed. We prove a similar black-boxing theorem for reaction networks whose dynamics are given by the non-linear rate equation. Along the way we describe a more general category of open dynamical systems where composition corresponds to gluing together open dynamical systems.

Time-Resolved Magneto-Optical Imaging of Superconducting YBCO Thin Films in the High-Frequency AC Current Regime

NASA Astrophysics Data System (ADS)

Frey, Alexander

We begin by defining the concept of `open' Markov processes, which are continuous-time Markov chains where probability can flow in and out through certain `boundary' states. We study open Markov processes which in the absence of such boundary flows admit equilibrium states satisfying detailed balance, meaning that the net flow of probability vanishes between all pairs of states. External couplings which fix the probabilities of boundary states can maintain such systems in non-equilibrium steady states in which non-zero probability currents flow. We show that these non-equilibrium steady states minimize a quadratic form which we call 'dissipation.' This is closely related to Prigogine's principle of minimum entropy production. We bound the rate of change of the entropy of a driven non-equilibrium steady state relative to the underlying equilibrium state in terms of the flow of probability through the boundary of the process. We then consider open Markov processes as morphisms in a symmetric monoidal category by splitting up their boundary states into certain sets of `inputs' and `outputs.' Composition corresponds to gluing the outputs of one such open Markov process onto the inputs of another so that the probability flowing out of the first process is equal to the probability flowing into the second. Tensoring in this category corresponds to placing two such systems side by side. We construct a `black-box' functor characterizing the behavior of an open Markov process in terms of the space of possible steady state probabilities and probability currents along the boundary. The fact that this is a functor means that the behavior of a composite open Markov process can be computed by composing the behaviors of the open Markov processes from which it is composed. We prove a similar black-boxing theorem for reaction networks whose dynamics are given by the non-linear rate equation. Along the way we describe a more general category of open dynamical systems where composition corresponds to gluing together open dynamical systems.

Use of Remote Sensing to Identify Essential Habitat for Aeschynomene virginica (L.) BSP, a Threatened Tidal Freshwater Wetland Plant

NASA Astrophysics Data System (ADS)

Mountz, Elizabeth M.

We begin by defining the concept of `open' Markov processes, which are continuous-time Markov chains where probability can flow in and out through certain `boundary' states. We study open Markov processes which in the absence of such boundary flows admit equilibrium states satisfying detailed balance, meaning that the net flow of probability vanishes between all pairs of states. External couplings which fix the probabilities of boundary states can maintain such systems in non-equilibrium steady states in which non-zero probability currents flow. We show that these non-equilibrium steady states minimize a quadratic form which we call 'dissipation.' This is closely related to Prigogine's principle of minimum entropy production. We bound the rate of change of the entropy of a driven non-equilibrium steady state relative to the underlying equilibrium state in terms of the flow of probability through the boundary of the process. We then consider open Markov processes as morphisms in a symmetric monoidal category by splitting up their boundary states into certain sets of `inputs' and `outputs.' Composition corresponds to gluing the outputs of one such open Markov process onto the inputs of another so that the probability flowing out of the first process is equal to the probability flowing into the second. Tensoring in this category corresponds to placing two such systems side by side. We construct a `black-box' functor characterizing the behavior of an open Markov process in terms of the space of possible steady state probabilities and probability currents along the boundary. The fact that this is a functor means that the behavior of a composite open Markov process can be computed by composing the behaviors of the open Markov processes from which it is composed. We prove a similar black-boxing theorem for reaction networks whose dynamics are given by the non-linear rate equation. Along the way we describe a more general category of open dynamical systems where composition corresponds to gluing together open dynamical systems.

Silver-Polyimide Nanocomposite Films: Single-Stage Synthesis and Analysis of Metalized Partially-Fluorinated Polyimide BTDA/4-BDAF Prepared from Silver(I) Complexes

NASA Astrophysics Data System (ADS)

Abelard, Joshua Erold Robert

We begin by defining the concept of `open' Markov processes, which are continuous-time Markov chains where probability can flow in and out through certain `boundary' states. We study open Markov processes which in the absence of such boundary flows admit equilibrium states satisfying detailed balance, meaning that the net flow of probability vanishes between all pairs of states. External couplings which fix the probabilities of boundary states can maintain such systems in non-equilibrium steady states in which non-zero probability currents flow. We show that these non-equilibrium steady states minimize a quadratic form which we call 'dissipation.' This is closely related to Prigogine's principle of minimum entropy production. We bound the rate of change of the entropy of a driven non-equilibrium steady state relative to the underlying equilibrium state in terms of the flow of probability through the boundary of the process. We then consider open Markov processes as morphisms in a symmetric monoidal category by splitting up their boundary states into certain sets of `inputs' and `outputs.' Composition corresponds to gluing the outputs of one such open Markov process onto the inputs of another so that the probability flowing out of the first process is equal to the probability flowing into the second. Tensoring in this category corresponds to placing two such systems side by side. We construct a `black-box' functor characterizing the behavior of an open Markov process in terms of the space of possible steady state probabilities and probability currents along the boundary. The fact that this is a functor means that the behavior of a composite open Markov process can be computed by composing the behaviors of the open Markov processes from which it is composed. We prove a similar black-boxing theorem for reaction networks whose dynamics are given by the non-linear rate equation. Along the way we describe a more general category of open dynamical systems where composition corresponds to gluing together open dynamical systems.

Multifunctional Polymer Synthesis and Incorporation of Gadolinium Compounds and Modified Tungsten Nanoparticles for Improvement of Radiation Shielding for use in Outer Space

NASA Astrophysics Data System (ADS)

Harbert, Emily Grace

We begin by defining the concept of `open' Markov processes, which are continuous-time Markov chains where probability can flow in and out through certain `boundary' states. We study open Markov processes which in the absence of such boundary flows admit equilibrium states satisfying detailed balance, meaning that the net flow of probability vanishes between all pairs of states. External couplings which fix the probabilities of boundary states can maintain such systems in non-equilibrium steady states in which non-zero probability currents flow. We show that these non-equilibrium steady states minimize a quadratic form which we call 'dissipation.' This is closely related to Prigogine's principle of minimum entropy production. We bound the rate of change of the entropy of a driven non-equilibrium steady state relative to the underlying equilibrium state in terms of the flow of probability through the boundary of the process. We then consider open Markov processes as morphisms in a symmetric monoidal category by splitting up their boundary states into certain sets of `inputs' and `outputs.' Composition corresponds to gluing the outputs of one such open Markov process onto the inputs of another so that the probability flowing out of the first process is equal to the probability flowing into the second. Tensoring in this category corresponds to placing two such systems side by side. We construct a `black-box' functor characterizing the behavior of an open Markov process in terms of the space of possible steady state probabilities and probability currents along the boundary. The fact that this is a functor means that the behavior of a composite open Markov process can be computed by composing the behaviors of the open Markov processes from which it is composed. We prove a similar black-boxing theorem for reaction networks whose dynamics are given by the non-linear rate equation. Along the way we describe a more general category of open dynamical systems where composition corresponds to gluing together open dynamical systems.

Model-Based Approach to Predict Adherence to Protocol During Antiobesity Trials.

PubMed

Sharma, Vishnu D; Combes, François P; Vakilynejad, Majid; Lahu, Gezim; Lesko, Lawrence J; Trame, Mirjam N

2018-02-01

Development of antiobesity drugs is continuously challenged by high dropout rates during clinical trials. The objective was to develop a population pharmacodynamic model that describes the temporal changes in body weight, considering disease progression, lifestyle intervention, and drug effects. Markov modeling (MM) was applied for quantification and characterization of responder and nonresponder as key drivers of dropout rates, to ultimately support the clinical trial simulations and the outcome in terms of trial adherence. Subjects (n = 4591) from 6 Contrave ® trials were included in this analysis. An indirect-response model developed by van Wart et al was used as a starting point. Inclusion of drug effect was dose driven using a population dose- and time-dependent pharmacodynamic (DTPD) model. Additionally, a population-pharmacokinetic parameter- and data (PPPD)-driven model was developed using the final DTPD model structure and final parameter estimates from a previously developed population pharmacokinetic model based on available Contrave ® pharmacokinetic concentrations. Last, MM was developed to predict transition rate probabilities among responder, nonresponder, and dropout states driven by the pharmacodynamic effect resulting from the DTPD or PPPD model. Covariates included in the models and parameters were diabetes mellitus and race. The linked DTPD-MM and PPPD-MM was able to predict transition rates among responder, nonresponder, and dropout states well. The analysis concluded that body-weight change is an important factor influencing dropout rates, and the MM depicted that overall a DTPD model-driven approach provides a reasonable prediction of clinical trial outcome probabilities similar to a pharmacokinetic-driven approach. © 2017, The Authors. The Journal of Clinical Pharmacology published by Wiley Periodicals, Inc. on behalf of American College of Clinical Pharmacology.

Multidimensional Latent Markov Models in a Developmental Study of Inhibitory Control and Attentional Flexibility in Early Childhood

ERIC Educational Resources Information Center

Bartolucci, Francesco; Solis-Trapala, Ivonne L.

2010-01-01

We demonstrate the use of a multidimensional extension of the latent Markov model to analyse data from studies with repeated binary responses in developmental psychology. In particular, we consider an experiment based on a battery of tests which was administered to pre-school children, at three time periods, in order to measure their inhibitory…

Evaluation of Markov-Decision Model for Instructional Sequence Optimization. Semi-Annual Technical Report for the period 1 July-31 December 1975. Technical Report No. 76.

ERIC Educational Resources Information Center

Wollmer, Richard D.; Bond, Nicholas A.

Two computer-assisted instruction programs were written in electronics and trigonometry to test the Wollmer Markov Model for optimizing hierarchial learning; calibration samples totalling 110 students completed these programs. Since the model postulated that transfer effects would be a function of the amount of practice, half of the students were…

Local Composite Quantile Regression Smoothing for Harris Recurrent Markov Processes

PubMed Central

Li, Degui; Li, Runze

2016-01-01

In this paper, we study the local polynomial composite quantile regression (CQR) smoothing method for the nonlinear and nonparametric models under the Harris recurrent Markov chain framework. The local polynomial CQR regression method is a robust alternative to the widely-used local polynomial method, and has been well studied in stationary time series. In this paper, we relax the stationarity restriction on the model, and allow that the regressors are generated by a general Harris recurrent Markov process which includes both the stationary (positive recurrent) and nonstationary (null recurrent) cases. Under some mild conditions, we establish the asymptotic theory for the proposed local polynomial CQR estimator of the mean regression function, and show that the convergence rate for the estimator in nonstationary case is slower than that in stationary case. Furthermore, a weighted type local polynomial CQR estimator is provided to improve the estimation efficiency, and a data-driven bandwidth selection is introduced to choose the optimal bandwidth involved in the nonparametric estimators. Finally, we give some numerical studies to examine the finite sample performance of the developed methodology and theory. PMID:27667894

Estimating parameters of hidden Markov models based on marked individuals: use of robust design data

USGS Publications Warehouse

Kendall, William L.; White, Gary C.; Hines, James E.; Langtimm, Catherine A.; Yoshizaki, Jun

2012-01-01

Development and use of multistate mark-recapture models, which provide estimates of parameters of Markov processes in the face of imperfect detection, have become common over the last twenty years. Recently, estimating parameters of hidden Markov models, where the state of an individual can be uncertain even when it is detected, has received attention. Previous work has shown that ignoring state uncertainty biases estimates of survival and state transition probabilities, thereby reducing the power to detect effects. Efforts to adjust for state uncertainty have included special cases and a general framework for a single sample per period of interest. We provide a flexible framework for adjusting for state uncertainty in multistate models, while utilizing multiple sampling occasions per period of interest to increase precision and remove parameter redundancy. These models also produce direct estimates of state structure for each primary period, even for the case where there is just one sampling occasion. We apply our model to expected value data, and to data from a study of Florida manatees, to provide examples of the improvement in precision due to secondary capture occasions. We also provide user-friendly software to implement these models. This general framework could also be used by practitioners to consider constrained models of particular interest, or model the relationship between within-primary period parameters (e.g., state structure) and between-primary period parameters (e.g., state transition probabilities).

Thermodynamically accurate modeling of the catalytic cycle of photosynthetic oxygen evolution: a mathematical solution to asymmetric Markov chains.

PubMed

Vinyard, David J; Zachary, Chase E; Ananyev, Gennady; Dismukes, G Charles

2013-07-01

Forty-three years ago, Kok and coworkers introduced a phenomenological model describing period-four oscillations in O2 flash yields during photosynthetic water oxidation (WOC), which had been first reported by Joliot and coworkers. The original two-parameter Kok model was subsequently extended in its level of complexity to better simulate diverse data sets, including intact cells and isolated PSII-WOCs, but at the expense of introducing physically unrealistic assumptions necessary to enable numerical solutions. To date, analytical solutions have been found only for symmetric Kok models (inefficiencies are equally probable for all intermediates, called "S-states"). However, it is widely accepted that S-state reaction steps are not identical and some are not reversible (by thermodynamic restraints) thereby causing asymmetric cycles. We have developed a mathematically more rigorous foundation that eliminates unphysical assumptions known to be in conflict with experiments and adopts a new experimental constraint on solutions. This new algorithm termed STEAMM for S-state Transition Eigenvalues of Asymmetric Markov Models enables solutions to models having fewer adjustable parameters and uses automated fitting to experimental data sets, yielding higher accuracy and precision than the classic Kok or extended Kok models. This new tool provides a general mathematical framework for analyzing damped oscillations arising from any cycle period using any appropriate Markov model, regardless of symmetry. We illustrate applications of STEAMM that better describe the intrinsic inefficiencies for photon-to-charge conversion within PSII-WOCs that are responsible for damped period-four and period-two oscillations of flash O2 yields across diverse species, while using simpler Markov models free from unrealistic assumptions. Copyright © 2013 Elsevier B.V. All rights reserved.

Alignment-free Transcriptomic and Metatranscriptomic Comparison Using Sequencing Signatures with Variable Length Markov Chains.

PubMed

Liao, Weinan; Ren, Jie; Wang, Kun; Wang, Shun; Zeng, Feng; Wang, Ying; Sun, Fengzhu

2016-11-23

The comparison between microbial sequencing data is critical to understand the dynamics of microbial communities. The alignment-based tools analyzing metagenomic datasets require reference sequences and read alignments. The available alignment-free dissimilarity approaches model the background sequences with Fixed Order Markov Chain (FOMC) yielding promising results for the comparison of microbial communities. However, in FOMC, the number of parameters grows exponentially with the increase of the order of Markov Chain (MC). Under a fixed high order of MC, the parameters might not be accurately estimated owing to the limitation of sequencing depth. In our study, we investigate an alternative to FOMC to model background sequences with the data-driven Variable Length Markov Chain (VLMC) in metatranscriptomic data. The VLMC originally designed for long sequences was extended to apply to high-throughput sequencing reads and the strategies to estimate the corresponding parameters were developed. The flexible number of parameters in VLMC avoids estimating the vast number of parameters of high-order MC under limited sequencing depth. Different from the manual selection in FOMC, VLMC determines the MC order adaptively. Several beta diversity measures based on VLMC were applied to compare the bacterial RNA-Seq and metatranscriptomic datasets. Experiments show that VLMC outperforms FOMC to model the background sequences in transcriptomic and metatranscriptomic samples. A software pipeline is available at https://d2vlmc.codeplex.com.

Modeling long correlation times using additive binary Markov chains: Applications to wind generation time series.

PubMed

Weber, Juliane; Zachow, Christopher; Witthaut, Dirk

2018-03-01

Wind power generation exhibits a strong temporal variability, which is crucial for system integration in highly renewable power systems. Different methods exist to simulate wind power generation but they often cannot represent the crucial temporal fluctuations properly. We apply the concept of additive binary Markov chains to model a wind generation time series consisting of two states: periods of high and low wind generation. The only input parameter for this model is the empirical autocorrelation function. The two-state model is readily extended to stochastically reproduce the actual generation per period. To evaluate the additive binary Markov chain method, we introduce a coarse model of the electric power system to derive backup and storage needs. We find that the temporal correlations of wind power generation, the backup need as a function of the storage capacity, and the resting time distribution of high and low wind events for different shares of wind generation can be reconstructed.

Modeling long correlation times using additive binary Markov chains: Applications to wind generation time series

NASA Astrophysics Data System (ADS)

Weber, Juliane; Zachow, Christopher; Witthaut, Dirk

2018-03-01

Wind power generation exhibits a strong temporal variability, which is crucial for system integration in highly renewable power systems. Different methods exist to simulate wind power generation but they often cannot represent the crucial temporal fluctuations properly. We apply the concept of additive binary Markov chains to model a wind generation time series consisting of two states: periods of high and low wind generation. The only input parameter for this model is the empirical autocorrelation function. The two-state model is readily extended to stochastically reproduce the actual generation per period. To evaluate the additive binary Markov chain method, we introduce a coarse model of the electric power system to derive backup and storage needs. We find that the temporal correlations of wind power generation, the backup need as a function of the storage capacity, and the resting time distribution of high and low wind events for different shares of wind generation can be reconstructed.

Population Synthesis of Radio and Y-ray Millisecond Pulsars Using Markov Chain Monte Carlo

NASA Astrophysics Data System (ADS)

Gonthier, Peter L.; Billman, C.; Harding, A. K.

2013-04-01

We present preliminary results of a new population synthesis of millisecond pulsars (MSP) from the Galactic disk using Markov Chain Monte Carlo techniques to better understand the model parameter space. We include empirical radio and γ-ray luminosity models that are dependent on the pulsar period and period derivative with freely varying exponents. The magnitudes of the model luminosities are adjusted to reproduce the number of MSPs detected by a group of ten radio surveys and by Fermi, predicting the MSP birth rate in the Galaxy. We follow a similar set of assumptions that we have used in previous, more constrained Monte Carlo simulations. The parameters associated with the birth distributions such as those for the accretion rate, magnetic field and period distributions are also free to vary. With the large set of free parameters, we employ Markov Chain Monte Carlo simulations to explore the large and small worlds of the parameter space. We present preliminary comparisons of the simulated and detected distributions of radio and γ-ray pulsar characteristics. We express our gratitude for the generous support of the National Science Foundation (REU and RUI), Fermi Guest Investigator Program and the NASA Astrophysics Theory and Fundamental Program.

Incorporating teleconnection information into reservoir operating policies using Stochastic Dynamic Programming and a Hidden Markov Model

NASA Astrophysics Data System (ADS)

Turner, Sean; Galelli, Stefano; Wilcox, Karen

2015-04-01

Water reservoir systems are often affected by recurring large-scale ocean-atmospheric anomalies, known as teleconnections, that cause prolonged periods of climatological drought. Accurate forecasts of these events -- at lead times in the order of weeks and months -- may enable reservoir operators to take more effective release decisions to improve the performance of their systems. In practice this might mean a more reliable water supply system, a more profitable hydropower plant or a more sustainable environmental release policy. To this end, climate indices, which represent the oscillation of the ocean-atmospheric system, might be gainfully employed within reservoir operating models that adapt the reservoir operation as a function of the climate condition. This study develops a Stochastic Dynamic Programming (SDP) approach that can incorporate climate indices using a Hidden Markov Model. The model simulates the climatic regime as a hidden state following a Markov chain, with the state transitions driven by variation in climatic indices, such as the Southern Oscillation Index. Time series analysis of recorded streamflow data reveals the parameters of separate autoregressive models that describe the inflow to the reservoir under three representative climate states ("normal", "wet", "dry"). These models then define inflow transition probabilities for use in a classic SDP approach. The key advantage of the Hidden Markov Model is that it allows conditioning the operating policy not only on the reservoir storage and the antecedent inflow, but also on the climate condition, thus potentially allowing adaptability to a broader range of climate conditions. In practice, the reservoir operator would effect a water release tailored to a specific climate state based on available teleconnection data and forecasts. The approach is demonstrated on the operation of a realistic, stylised water reservoir with carry-over capacity in South-East Australia. Here teleconnections relating to both the El Niño Southern Oscillation and the Indian Ocean Dipole influence local hydro-meteorological processes; statistically significant lag correlations have already been established. Simulation of the derived operating policies, which are benchmarked against standard policies conditioned on the reservoir storage and the antecedent inflow, demonstrates the potential of the proposed approach. Future research will further develop the model for sensitivity analysis and regional studies examining the economic value of incorporating long range forecasts into reservoir operation.

Estimation of sojourn time in chronic disease screening without data on interval cases.

PubMed

Chen, T H; Kuo, H S; Yen, M F; Lai, M S; Tabar, L; Duffy, S W

2000-03-01

Estimation of the sojourn time on the preclinical detectable period in disease screening or transition rates for the natural history of chronic disease usually rely on interval cases (diagnosed between screens). However, to ascertain such cases might be difficult in developing countries due to incomplete registration systems and difficulties in follow-up. To overcome this problem, we propose three Markov models to estimate parameters without using interval cases. A three-state Markov model, a five-state Markov model related to regional lymph node spread, and a five-state Markov model pertaining to tumor size are applied to data on breast cancer screening in female relatives of breast cancer cases in Taiwan. Results based on a three-state Markov model give mean sojourn time (MST) 1.90 (95% CI: 1.18-4.86) years for this high-risk group. Validation of these models on the basis of data on breast cancer screening in the age groups 50-59 and 60-69 years from the Swedish Two-County Trial shows the estimates from a three-state Markov model that does not use interval cases are very close to those from previous Markov models taking interval cancers into account. For the five-state Markov model, a reparameterized procedure using auxiliary information on clinically detected cancers is performed to estimate relevant parameters. A good fit of internal and external validation demonstrates the feasibility of using these models to estimate parameters that have previously required interval cancers. This method can be applied to other screening data in which there are no data on interval cases.

Real-time antenna fault diagnosis experiments at DSS 13

NASA Technical Reports Server (NTRS)

Mellstrom, J.; Pierson, C.; Smyth, P.

1992-01-01

Experimental results obtained when a previously described fault diagnosis system was run online in real time at the 34-m beam waveguide antenna at Deep Space Station (DSS) 13 are described. Experimental conditions and the quality of results are described. A neural network model and a maximum-likelihood Gaussian classifier are compared with and without a Markov component to model temporal context. At the rate of a state update every 6.4 seconds, over a period of roughly 1 hour, the neural-Markov system had zero errors (incorrect state estimates) while monitoring both faulty and normal operations. The overall results indicate that the neural-Markov combination is the most accurate model and has significant practical potential.

Information-Theoretic Performance Analysis of Sensor Networks via Markov Modeling of Time Series Data.

PubMed

Li, Yue; Jha, Devesh K; Ray, Asok; Wettergren, Thomas A; Yue Li; Jha, Devesh K; Ray, Asok; Wettergren, Thomas A; Wettergren, Thomas A; Li, Yue; Ray, Asok; Jha, Devesh K

2018-06-01

This paper presents information-theoretic performance analysis of passive sensor networks for detection of moving targets. The proposed method falls largely under the category of data-level information fusion in sensor networks. To this end, a measure of information contribution for sensors is formulated in a symbolic dynamics framework. The network information state is approximately represented as the largest principal component of the time series collected across the network. To quantify each sensor's contribution for generation of the information content, Markov machine models as well as x-Markov (pronounced as cross-Markov) machine models, conditioned on the network information state, are constructed; the difference between the conditional entropies of these machines is then treated as an approximate measure of information contribution by the respective sensors. The x-Markov models represent the conditional temporal statistics given the network information state. The proposed method has been validated on experimental data collected from a local area network of passive sensors for target detection, where the statistical characteristics of environmental disturbances are similar to those of the target signal in the sense of time scale and texture. A distinctive feature of the proposed algorithm is that the network decisions are independent of the behavior and identity of the individual sensors, which is desirable from computational perspectives. Results are presented to demonstrate the proposed method's efficacy to correctly identify the presence of a target with very low false-alarm rates. The performance of the underlying algorithm is compared with that of a recent data-driven, feature-level information fusion algorithm. It is shown that the proposed algorithm outperforms the other algorithm.

Nonequilibrium steady states of ideal bosonic and fermionic quantum gases.

PubMed

Vorberg, Daniel; Wustmann, Waltraut; Schomerus, Henning; Ketzmerick, Roland; Eckardt, André

2015-12-01

We investigate nonequilibrium steady states of driven-dissipative ideal quantum gases of both bosons and fermions. We focus on systems of sharp particle number that are driven out of equilibrium either by the coupling to several heat baths of different temperature or by time-periodic driving in combination with the coupling to a heat bath. Within the framework of (Floquet-)Born-Markov theory, several analytical and numerical methods are described in detail. This includes a mean-field theory in terms of occupation numbers, an augmented mean-field theory taking into account also nontrivial two-particle correlations, and quantum-jump-type Monte Carlo simulations. For the case of the ideal Fermi gas, these methods are applied to simple lattice models and the possibility of achieving exotic states via bath engineering is pointed out. The largest part of this work is devoted to bosonic quantum gases and the phenomenon of Bose selection, a nonequilibrium generalization of Bose condensation, where multiple single-particle states are selected to acquire a large occupation [Phys. Rev. Lett. 111, 240405 (2013)]. In this context, among others, we provide a theory for transitions where the set of selected states changes, describe an efficient algorithm for finding the set of selected states, investigate beyond-mean-field effects, and identify the dominant mechanisms for heat transport in the Bose-selected state.

Assessing the Deterrence Value of Carrier Presence Against Adversary Aggression in a Coalition Environment

DTIC Science & Technology

2017-09-01

seeks to quantify the deterrence value of a CSG using a game - theoretic framework. Consider a region with several nations, where two major players stand...develop a Markov game to model the interactions between the two players and these other nations over a period of time. The game starts in Notional...establishing diplomatic advantage are equally important in deterring aggression. 14. SUBJECT TERMS carrier strike group, CSG, deterrence, Markov game

Slow diffusion by Markov random flights

NASA Astrophysics Data System (ADS)

Kolesnik, Alexander D.

2018-06-01

We present a conception of the slow diffusion processes in the Euclidean spaces Rm , m ≥ 1, based on the theory of random flights with small constant speed that are driven by a homogeneous Poisson process of small rate. The slow diffusion condition that, on long time intervals, leads to the stationary distributions, is given. The stationary distributions of slow diffusion processes in some Euclidean spaces of low dimensions, are presented.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Ullah, Ghanim; Demuro, Angelo; Parker, Ian

Amyloid beta (Aβ) oligomers associated with Alzheimer’s disease (AD) form Ca 2+-permeable plasma membrane pores, leading to a disruption of the otherwise well-controlled intracellular calcium (Ca 2+) homeostasis. The resultant up-regulation of intracellular Ca 2+ concentration has detrimental implications for memory formation and cell survival. The gating kinetics and Ca 2+ permeability of Aβ pores are not well understood. We have used computational modeling in conjunction with the ability of optical patch-clamping for massively parallel imaging of Ca 2+ flux through thousands of pores in the cell membrane of Xenopus oocytes to elucidate the kinetic properties of Aβ pores. Themore » fluorescence time-series data from individual pores were idealized and used to develop data-driven Markov chain models for the kinetics of the Aβ pore at different stages of its evolution. Our study provides the first demonstration of developing Markov chain models for ion channel gating that are driven by optical-patch clamp data with the advantage of experiments being performed under close to physiological conditions. As a result, we demonstrate the up-regulation of gating of various Ca 2+ release channels due to Aβ pores and show that the extent and spatial range of such up-regulation increases as Aβ pores with low open probability and Ca 2+ permeability transition into those with high open probability and Ca 2+ permeability.« less

Machine Learning

DOE Office of Scientific and Technical Information (OSTI.GOV)

Chikkagoudar, Satish; Chatterjee, Samrat; Thomas, Dennis G.

The absence of a robust and unified theory of cyber dynamics presents challenges and opportunities for using machine learning based data-driven approaches to further the understanding of the behavior of such complex systems. Analysts can also use machine learning approaches to gain operational insights. In order to be operationally beneficial, cybersecurity machine learning based models need to have the ability to: (1) represent a real-world system, (2) infer system properties, and (3) learn and adapt based on expert knowledge and observations. Probabilistic models and Probabilistic graphical models provide these necessary properties and are further explored in this chapter. Bayesian Networksmore » and Hidden Markov Models are introduced as an example of a widely used data driven classification/modeling strategy.« less

Global-constrained hidden Markov model applied on wireless capsule endoscopy video segmentation

NASA Astrophysics Data System (ADS)

Wan, Yiwen; Duraisamy, Prakash; Alam, Mohammad S.; Buckles, Bill

2012-06-01

Accurate analysis of wireless capsule endoscopy (WCE) videos is vital but tedious. Automatic image analysis can expedite this task. Video segmentation of WCE into the four parts of the gastrointestinal tract is one way to assist a physician. The segmentation approach described in this paper integrates pattern recognition with statiscal analysis. Iniatially, a support vector machine is applied to classify video frames into four classes using a combination of multiple color and texture features as the feature vector. A Poisson cumulative distribution, for which the parameter depends on the length of segments, models a prior knowledge. A priori knowledge together with inter-frame difference serves as the global constraints driven by the underlying observation of each WCE video, which is fitted by Gaussian distribution to constrain the transition probability of hidden Markov model.Experimental results demonstrated effectiveness of the approach.

Resolvent-Techniques for Multiple Exercise Problems

DOE Office of Scientific and Technical Information (OSTI.GOV)

Christensen, Sören, E-mail: christensen@math.uni-kiel.de; Lempa, Jukka, E-mail: jukka.lempa@hioa.no

2015-02-15

We study optimal multiple stopping of strong Markov processes with random refraction periods. The refraction periods are assumed to be exponentially distributed with a common rate and independent of the underlying dynamics. Our main tool is using the resolvent operator. In the first part, we reduce infinite stopping problems to ordinary ones in a general strong Markov setting. This leads to explicit solutions for wide classes of such problems. Starting from this result, we analyze problems with finitely many exercise rights and explain solution methods for some classes of problems with underlying Lévy and diffusion processes, where the optimal characteristicsmore » of the problems can be identified more explicitly. We illustrate the main results with explicit examples.« less

Population synthesis of radio and gamma-ray millisecond pulsars using Markov Chain Monte Carlo techniques

NASA Astrophysics Data System (ADS)

Gonthier, Peter L.; Koh, Yew-Meng; Kust Harding, Alice

2016-04-01

We present preliminary results of a new population synthesis of millisecond pulsars (MSP) from the Galactic disk using Markov Chain Monte Carlo techniques to better understand the model parameter space. We include empirical radio and gamma-ray luminosity models that are dependent on the pulsar period and period derivative with freely varying exponents. The magnitudes of the model luminosities are adjusted to reproduce the number of MSPs detected by a group of thirteen radio surveys as well as the MSP birth rate in the Galaxy and the number of MSPs detected by Fermi. We explore various high-energy emission geometries like the slot gap, outer gap, two pole caustic and pair starved polar cap models. The parameters associated with the birth distributions for the mass accretion rate, magnetic field, and period distributions are well constrained. With the set of four free parameters, we employ Markov Chain Monte Carlo simulations to explore the model parameter space. We present preliminary comparisons of the simulated and detected distributions of radio and gamma-ray pulsar characteristics. We estimate the contribution of MSPs to the diffuse gamma-ray background with a special focus on the Galactic Center.We express our gratitude for the generous support of the National Science Foundation (RUI: AST-1009731), Fermi Guest Investigator Program and the NASA Astrophysics Theory and Fundamental Program (NNX09AQ71G).

Development of reversible jump Markov Chain Monte Carlo algorithm in the Bayesian mixture modeling for microarray data in Indonesia

NASA Astrophysics Data System (ADS)

Astuti, Ani Budi; Iriawan, Nur; Irhamah, Kuswanto, Heri

2017-12-01

In the Bayesian mixture modeling requires stages the identification number of the most appropriate mixture components thus obtained mixture models fit the data through data driven concept. Reversible Jump Markov Chain Monte Carlo (RJMCMC) is a combination of the reversible jump (RJ) concept and the Markov Chain Monte Carlo (MCMC) concept used by some researchers to solve the problem of identifying the number of mixture components which are not known with certainty number. In its application, RJMCMC using the concept of the birth/death and the split-merge with six types of movement, that are w updating, θ updating, z updating, hyperparameter β updating, split-merge for components and birth/death from blank components. The development of the RJMCMC algorithm needs to be done according to the observed case. The purpose of this study is to know the performance of RJMCMC algorithm development in identifying the number of mixture components which are not known with certainty number in the Bayesian mixture modeling for microarray data in Indonesia. The results of this study represent that the concept RJMCMC algorithm development able to properly identify the number of mixture components in the Bayesian normal mixture model wherein the component mixture in the case of microarray data in Indonesia is not known for certain number.

Nonequilibrium steady states of ideal bosonic and fermionic quantum gases

NASA Astrophysics Data System (ADS)

Vorberg, Daniel; Wustmann, Waltraut; Schomerus, Henning; Ketzmerick, Roland; Eckardt, André

2015-12-01

We investigate nonequilibrium steady states of driven-dissipative ideal quantum gases of both bosons and fermions. We focus on systems of sharp particle number that are driven out of equilibrium either by the coupling to several heat baths of different temperature or by time-periodic driving in combination with the coupling to a heat bath. Within the framework of (Floquet-)Born-Markov theory, several analytical and numerical methods are described in detail. This includes a mean-field theory in terms of occupation numbers, an augmented mean-field theory taking into account also nontrivial two-particle correlations, and quantum-jump-type Monte Carlo simulations. For the case of the ideal Fermi gas, these methods are applied to simple lattice models and the possibility of achieving exotic states via bath engineering is pointed out. The largest part of this work is devoted to bosonic quantum gases and the phenomenon of Bose selection, a nonequilibrium generalization of Bose condensation, where multiple single-particle states are selected to acquire a large occupation [Phys. Rev. Lett. 111, 240405 (2013), 10.1103/PhysRevLett.111.240405]. In this context, among others, we provide a theory for transitions where the set of selected states changes, describe an efficient algorithm for finding the set of selected states, investigate beyond-mean-field effects, and identify the dominant mechanisms for heat transport in the Bose-selected state.

HMM-based lexicon-driven and lexicon-free word recognition for online handwritten Indic scripts.

PubMed

Bharath, A; Madhvanath, Sriganesh

2012-04-01

Research for recognizing online handwritten words in Indic scripts is at its early stages when compared to Latin and Oriental scripts. In this paper, we address this problem specifically for two major Indic scripts--Devanagari and Tamil. In contrast to previous approaches, the techniques we propose are largely data driven and script independent. We propose two different techniques for word recognition based on Hidden Markov Models (HMM): lexicon driven and lexicon free. The lexicon-driven technique models each word in the lexicon as a sequence of symbol HMMs according to a standard symbol writing order derived from the phonetic representation. The lexicon-free technique uses a novel Bag-of-Symbols representation of the handwritten word that is independent of symbol order and allows rapid pruning of the lexicon. On handwritten Devanagari word samples featuring both standard and nonstandard symbol writing orders, a combination of lexicon-driven and lexicon-free recognizers significantly outperforms either of them used in isolation. In contrast, most Tamil word samples feature the standard symbol order, and the lexicon-driven recognizer outperforms the lexicon free one as well as their combination. The best recognition accuracies obtained for 20,000 word lexicons are 87.13 percent for Devanagari when the two recognizers are combined, and 91.8 percent for Tamil using the lexicon-driven technique.

Pacing of Paleozoic macroevolutionary rates by Milankovitch grand cycles.

PubMed

Crampton, James S; Meyers, Stephen R; Cooper, Roger A; Sadler, Peter M; Foote, Michael; Harte, David

2018-05-29

Periodic fluctuations in past biodiversity, speciation, and extinction have been proposed, with extremely long periods ranging from 26 to 62 million years, although forcing mechanisms remain speculative. In contrast, well-understood periodic Milankovitch climate forcing represents a viable driver for macroevolutionary fluctuations, although little evidence for such fluctuation exists except during the Late Cenozoic. The reality, magnitude, and drivers of periodic fluctuations in macroevolutionary rates are of interest given long-standing debate surrounding the relative roles of intrinsic biotic interactions vs. extrinsic environmental factors as drivers of biodiversity change. Here, we show that, over a time span of 60 million years, between 9 and 16% of the variance in biological turnover (i.e., speciation probability plus species extinction probability) in a major Early Paleozoic zooplankton group, the graptoloids, can be explained by long-period astronomical cycles (Milankovitch "grand cycles") associated with Earth's orbital eccentricity (2.6 million years) and obliquity (1.3 million years). These grand cycles modulate climate variability, alternating times of relative stability in the environment with times of maximum volatility. We infer that these cycles influenced graptolite speciation and extinction through climate-driven changes to oceanic circulation and structure. Our results confirm the existence of Milankovitch grand cycles in the Early Paleozoic Era and show that known processes related to the mechanics of the Solar System were shaping marine macroevolutionary rates comparatively early in the history of complex life. We present an application of hidden Markov models to macroevolutionary time series and protocols for the evaluation of statistical significance in spectral analysis.

Analyzing and modeling the kinetics of amyloid beta pores associated with Alzheimer’s disease pathology

DOE PAGES

Ullah, Ghanim; Demuro, Angelo; Parker, Ian; ...

2015-09-08

Amyloid beta (Aβ) oligomers associated with Alzheimer’s disease (AD) form Ca 2+-permeable plasma membrane pores, leading to a disruption of the otherwise well-controlled intracellular calcium (Ca 2+) homeostasis. The resultant up-regulation of intracellular Ca 2+ concentration has detrimental implications for memory formation and cell survival. The gating kinetics and Ca 2+ permeability of Aβ pores are not well understood. We have used computational modeling in conjunction with the ability of optical patch-clamping for massively parallel imaging of Ca 2+ flux through thousands of pores in the cell membrane of Xenopus oocytes to elucidate the kinetic properties of Aβ pores. Themore » fluorescence time-series data from individual pores were idealized and used to develop data-driven Markov chain models for the kinetics of the Aβ pore at different stages of its evolution. Our study provides the first demonstration of developing Markov chain models for ion channel gating that are driven by optical-patch clamp data with the advantage of experiments being performed under close to physiological conditions. As a result, we demonstrate the up-regulation of gating of various Ca 2+ release channels due to Aβ pores and show that the extent and spatial range of such up-regulation increases as Aβ pores with low open probability and Ca 2+ permeability transition into those with high open probability and Ca 2+ permeability.« less

Economic modeling of new stent platforms to evaluate cost effectiveness: analysis of the TAXUS Liberté versus TAXUS express stents.

PubMed

Turco, Mark A; Kansal, Anuraag R; Stern, Sean; Amorosi, Stacey L; Underwood, Paul L; Lissovoy, Greg D E; Dawkins, Keith D

2012-08-01

With the changing health care environment, cost effectiveness is an important adjunct to clinical investigation when assessing new medical devices. This study presents an economic model to evaluate cost effectiveness of coronary stents. Markov modeling was developed comparing total costs (Medicare payer perspective) between TAXUS Liberté and TAXUS Express based on 3-year clinical outcomes from the TAXUS ATLAS Small Vessel and Long Lesion trials. The TAXUS Liberté 2.25-mm stent provided cost savings relative to TAXUS Express from a payer perspective ($17,605 vs. $20,281), driven by reduced target vessel revascularization (0.16 events/patient vs. 0.33 events/patient). In probabilistic sensitivity analyses, TAXUS Liberté was less costly with fewer major adverse cardiac events in over 99% of parameter sets. The TAXUS Liberté Long (38 mm) stent was cost neutral relative to TAXUS Express from a payer perspective ($18,545 vs. $18,551) with fewer myocardial infarctions and cardiac deaths. Accounting for angiography-driven revascularizations, TAXUS Liberté 2.25 mm still provided cost savings relative to TAXUS Express ($16,822 vs. $19,139), although TAXUS Liberté Long was more expensive than TAXUS Express ($17,886 vs. $17,652). From a hospital perspective, TAXUS Liberté Long provided cost savings up to a price premium of $671/stent, driven by fewer stents employed per patient. This analysis confirms the utility of economic modeling in assessing new stent platforms. TAXUS Liberté 2.25 mm is economically dominant relative to TAXUS Express when treating small vessels. TAXUS Liberté Long is cost neutral to modestly more costly than TAXUS Express 2.25 mm from a payer perspective. ©2012, Wiley Periodicals, Inc.

Reciprocal Sliding Friction Model for an Electro-Deposited Coating and Its Parameter Estimation Using Markov Chain Monte Carlo Method

PubMed Central

Kim, Kyungmok; Lee, Jaewook

2016-01-01

This paper describes a sliding friction model for an electro-deposited coating. Reciprocating sliding tests using ball-on-flat plate test apparatus are performed to determine an evolution of the kinetic friction coefficient. The evolution of the friction coefficient is classified into the initial running-in period, steady-state sliding, and transition to higher friction. The friction coefficient during the initial running-in period and steady-state sliding is expressed as a simple linear function. The friction coefficient in the transition to higher friction is described with a mathematical model derived from Kachanov-type damage law. The model parameters are then estimated using the Markov Chain Monte Carlo (MCMC) approach. It is identified that estimated friction coefficients obtained by MCMC approach are in good agreement with measured ones. PMID:28773359

Generalized species sampling priors with latent Beta reinforcements

PubMed Central

Airoldi, Edoardo M.; Costa, Thiago; Bassetti, Federico; Leisen, Fabrizio; Guindani, Michele

2014-01-01

Many popular Bayesian nonparametric priors can be characterized in terms of exchangeable species sampling sequences. However, in some applications, exchangeability may not be appropriate. We introduce a novel and probabilistically coherent family of non-exchangeable species sampling sequences characterized by a tractable predictive probability function with weights driven by a sequence of independent Beta random variables. We compare their theoretical clustering properties with those of the Dirichlet Process and the two parameters Poisson-Dirichlet process. The proposed construction provides a complete characterization of the joint process, differently from existing work. We then propose the use of such process as prior distribution in a hierarchical Bayes modeling framework, and we describe a Markov Chain Monte Carlo sampler for posterior inference. We evaluate the performance of the prior and the robustness of the resulting inference in a simulation study, providing a comparison with popular Dirichlet Processes mixtures and Hidden Markov Models. Finally, we develop an application to the detection of chromosomal aberrations in breast cancer by leveraging array CGH data. PMID:25870462

Copula-based prediction of economic movements

NASA Astrophysics Data System (ADS)

García, J. E.; González-López, V. A.; Hirsh, I. D.

2016-06-01

In this paper we model the discretized returns of two paired time series BM&FBOVESPA Dividend Index and BM&FBOVESPA Public Utilities Index using multivariate Markov models. The discretization corresponds to three categories, high losses, high profits and the complementary periods of the series. In technical terms, the maximal memory that can be considered for a Markov model, can be derived from the size of the alphabet and dataset. The number of parameters needed to specify a discrete multivariate Markov chain grows exponentially with the order and dimension of the chain. In this case the size of the database is not large enough for a consistent estimation of the model. We apply a strategy to estimate a multivariate process with an order greater than the order achieved using standard procedures. The new strategy consist on obtaining a partition of the state space which is constructed from a combination, of the partitions corresponding to the two marginal processes and the partition corresponding to the multivariate Markov chain. In order to estimate the transition probabilities, all the partitions are linked using a copula. In our application this strategy provides a significant improvement in the movement predictions.

Variance-reduced simulation of lattice discrete-time Markov chains with applications in reaction networks

NASA Astrophysics Data System (ADS)

Maginnis, P. A.; West, M.; Dullerud, G. E.

2016-10-01

We propose an algorithm to accelerate Monte Carlo simulation for a broad class of stochastic processes. Specifically, the class of countable-state, discrete-time Markov chains driven by additive Poisson noise, or lattice discrete-time Markov chains. In particular, this class includes simulation of reaction networks via the tau-leaping algorithm. To produce the speedup, we simulate pairs of fair-draw trajectories that are negatively correlated. Thus, when averaged, these paths produce an unbiased Monte Carlo estimator that has reduced variance and, therefore, reduced error. Numerical results for three example systems included in this work demonstrate two to four orders of magnitude reduction of mean-square error. The numerical examples were chosen to illustrate different application areas and levels of system complexity. The areas are: gene expression (affine state-dependent rates), aerosol particle coagulation with emission and human immunodeficiency virus infection (both with nonlinear state-dependent rates). Our algorithm views the system dynamics as a ;black-box;, i.e., we only require control of pseudorandom number generator inputs. As a result, typical codes can be retrofitted with our algorithm using only minor changes. We prove several analytical results. Among these, we characterize the relationship of covariances between paths in the general nonlinear state-dependent intensity rates case, and we prove variance reduction of mean estimators in the special case of affine intensity rates.

Zero-state Markov switching count-data models: an empirical assessment.

PubMed

Malyshkina, Nataliya V; Mannering, Fred L

2010-01-01

In this study, a two-state Markov switching count-data model is proposed as an alternative to zero-inflated models to account for the preponderance of zeros sometimes observed in transportation count data, such as the number of accidents occurring on a roadway segment over some period of time. For this accident-frequency case, zero-inflated models assume the existence of two states: one of the states is a zero-accident count state, which has accident probabilities that are so low that they cannot be statistically distinguished from zero, and the other state is a normal-count state, in which counts can be non-negative integers that are generated by some counting process, for example, a Poisson or negative binomial. While zero-inflated models have come under some criticism with regard to accident-frequency applications - one fact is undeniable - in many applications they provide a statistically superior fit to the data. The Markov switching approach we propose seeks to overcome some of the criticism associated with the zero-accident state of the zero-inflated model by allowing individual roadway segments to switch between zero and normal-count states over time. An important advantage of this Markov switching approach is that it allows for the direct statistical estimation of the specific roadway-segment state (i.e., zero-accident or normal-count state) whereas traditional zero-inflated models do not. To demonstrate the applicability of this approach, a two-state Markov switching negative binomial model (estimated with Bayesian inference) and standard zero-inflated negative binomial models are estimated using five-year accident frequencies on Indiana interstate highway segments. It is shown that the Markov switching model is a viable alternative and results in a superior statistical fit relative to the zero-inflated models.

Bacterial genomes lacking long-range correlations may not be modeled by low-order Markov chains: the role of mixing statistics and frame shift of neighboring genes.

PubMed

Cocho, Germinal; Miramontes, Pedro; Mansilla, Ricardo; Li, Wentian

2014-12-01

We examine the relationship between exponential correlation functions and Markov models in a bacterial genome in detail. Despite the well known fact that Markov models generate sequences with correlation function that decays exponentially, simply constructed Markov models based on nearest-neighbor dimer (first-order), trimer (second-order), up to hexamer (fifth-order), and treating the DNA sequence as being homogeneous all fail to predict the value of exponential decay rate. Even reading-frame-specific Markov models (both first- and fifth-order) could not explain the fact that the exponential decay is very slow. Starting with the in-phase coding-DNA-sequence (CDS), we investigated correlation within a fixed-codon-position subsequence, and in artificially constructed sequences by packing CDSs with out-of-phase spacers, as well as altering CDS length distribution by imposing an upper limit. From these targeted analyses, we conclude that the correlation in the bacterial genomic sequence is mainly due to a mixing of heterogeneous statistics at different codon positions, and the decay of correlation is due to the possible out-of-phase between neighboring CDSs. There are also small contributions to the correlation from bases at the same codon position, as well as by non-coding sequences. These show that the seemingly simple exponential correlation functions in bacterial genome hide a complexity in correlation structure which is not suitable for a modeling by Markov chain in a homogeneous sequence. Other results include: use of the (absolute value) second largest eigenvalue to represent the 16 correlation functions and the prediction of a 10-11 base periodicity from the hexamer frequencies. Copyright © 2014 Elsevier Ltd. All rights reserved.

Analysis of Land Use and Land Cover Changes and Their Impacts on Future Runoff in the Luanhe River Basin in North China Using Markov and SWAT

NASA Astrophysics Data System (ADS)

Yang, W.; Long, D.

2017-12-01

Both land use/cover change (LUCC) and climate change exert significant impacts on runoff, which needs to be thoroughly examined in the context of urbanization, population growth, and climate change. The majority of studies focus on the impacts of either LUCC or climate on runoff in the upper reaches of the Panjiakou Reservoir in the Luanhe River basin, North China. In this study, first, two land use change matrices for periods 1970‒1980 and 1980‒2000 were constructed based on the theory of the Markov Chain which were used to predict the land use scenario of the basin in year 2020. Second, a distributed hydrological model, Soil Water Assessment Tools (SWAT), was set up and driven mainly by the China Gauge-based Daily Precipitation Analysis (CGDPA) product and outputs from three general circulation models (GCMs) of the Inter-Sectoral Impact Model Inter-comparison Project (ISI-MIP). Third, under the land use scenario in 2000, streamflow at the Chengde gauging station for the period 1998‒2014 was simulated with the CGDPA as input, and streamflow for the period 2015‒2025 under four representative concentration pathways (RCPs) was simulated using the outputs from GCMs and compared under the land use scenarios in 2000 and 2020. Results show that during 2015‒2025, the ensemble average precipitation in summer (i.e., from June to August) may increase up to 20% but decrease by -16% in fall (i.e., from September to November). The streamflow may increase in all the seasons, particularly in spring (i.e., from March to May) and summer reaching 150% and 142%, respectively. Furthermore, the streamflow may increase even more when the land use scenario for the period 1998‒2025 remains the same as that in 2000. The minimum (61mm) and maximum (77mm) mean annual runoff depth occur under the RCP4.5 and RCP6 scenarios, respectively, compared with the mean annual observed streamflow of 33 mm from 1998 to 2014. Finally, we analyzed the correlation among the main land use types (i.e., agricultural land, forest, and pasture) and evapotranspiration, surface runoff contribution to streamflow (SURQ), groundwater contribution to streamflow (GWQ), and the sum of the surface runoff and groundwater contributions to streamflow (SSGQ), respectively. It was found that the increase in agricultural land may induce the increase in SURQ but the decrease in GWQ.

Optimal firm growth under the threat of entry

PubMed Central

Kort, Peter M.; Wrzaczek, Stefan

2015-01-01

The paper studies the incumbent-entrant problem in a fully dynamic setting. We find that under an open-loop information structure the incumbent anticipates entry by overinvesting, whereas in the Markov perfect equilibrium the incumbent slightly underinvests in the period before the entry. The entry cost level where entry accommodation passes into entry deterrence is lower in the Markov perfect equilibrium. Further we find that the incumbent’s capital stock level needed to deter entry is hump shaped as a function of the entry time, whereas the corresponding entry cost, where the entrant is indifferent between entry and non-entry, is U-shaped. PMID:26435573

A Hidden Markov Model for Analysis of Frontline Veterinary Data for Emerging Zoonotic Disease Surveillance

PubMed Central

Robertson, Colin; Sawford, Kate; Gunawardana, Walimunige S. N.; Nelson, Trisalyn A.; Nathoo, Farouk; Stephen, Craig

2011-01-01

Surveillance systems tracking health patterns in animals have potential for early warning of infectious disease in humans, yet there are many challenges that remain before this can be realized. Specifically, there remains the challenge of detecting early warning signals for diseases that are not known or are not part of routine surveillance for named diseases. This paper reports on the development of a hidden Markov model for analysis of frontline veterinary sentinel surveillance data from Sri Lanka. Field veterinarians collected data on syndromes and diagnoses using mobile phones. A model for submission patterns accounts for both sentinel-related and disease-related variability. Models for commonly reported cattle diagnoses were estimated separately. Region-specific weekly average prevalence was estimated for each diagnoses and partitioned into normal and abnormal periods. Visualization of state probabilities was used to indicate areas and times of unusual disease prevalence. The analysis suggests that hidden Markov modelling is a useful approach for surveillance datasets from novel populations and/or having little historical baselines. PMID:21949763

Semi-Markov models for interval censored transient cognitive states with back transitions and a competing risk

PubMed Central

Wei, Shaoceng; Kryscio, Richard J.

2015-01-01

Continuous-time multi-state stochastic processes are useful for modeling the flow of subjects from intact cognition to dementia with mild cognitive impairment and global impairment as intervening transient, cognitive states and death as a competing risk (Figure 1). Each subject's cognition is assessed periodically resulting in interval censoring for the cognitive states while death without dementia is not interval censored. Since back transitions among the transient states are possible, Markov chains are often applied to this type of panel data. In this manuscript we apply a Semi-Markov process in which we assume that the waiting times are Weibull distributed except for transitions from the baseline state, which are exponentially distributed and in which we assume no additional changes in cognition occur between two assessments. We implement a quasi-Monte Carlo (QMC) method to calculate the higher order integration needed for likelihood estimation. We apply our model to a real dataset, the Nun Study, a cohort of 461 participants. PMID:24821001

Semi-Markov models for interval censored transient cognitive states with back transitions and a competing risk.

PubMed

Wei, Shaoceng; Kryscio, Richard J

2016-12-01

Continuous-time multi-state stochastic processes are useful for modeling the flow of subjects from intact cognition to dementia with mild cognitive impairment and global impairment as intervening transient cognitive states and death as a competing risk. Each subject's cognition is assessed periodically resulting in interval censoring for the cognitive states while death without dementia is not interval censored. Since back transitions among the transient states are possible, Markov chains are often applied to this type of panel data. In this manuscript, we apply a semi-Markov process in which we assume that the waiting times are Weibull distributed except for transitions from the baseline state, which are exponentially distributed and in which we assume no additional changes in cognition occur between two assessments. We implement a quasi-Monte Carlo (QMC) method to calculate the higher order integration needed for likelihood estimation. We apply our model to a real dataset, the Nun Study, a cohort of 461 participants. © The Author(s) 2014.

Twelve years of succession on sandy substrates in a post-mining landscape: a Markov chain analysis.

PubMed

Baasch, Annett; Tischew, Sabine; Bruelheide, Helge

2010-06-01

Knowledge of succession rates and pathways is crucial for devising restoration strategies for highly disturbed ecosystems such as surface-mined land. As these processes have often only been described in qualitative terms, we used Markov models to quantify transitions between successional stages. However, Markov models are often considered not attractive for some reasons, such as model assumptions (e.g., stationarity in space and time, or the high expenditure of time required to estimate successional transitions in the field). Here we present a solution for converting multivariate ecological time series into transition matrices and demonstrate the applicability of this approach for a data set that resulted from monitoring the succession of sandy dry grassland in a post-mining landscape. We analyzed five transition matrices, four one-step matrices referring to specific periods of transition (1995-1998, 1998-2001, 2001-2004, 2004-2007), and one matrix for the whole study period (stationary model, 1995-2007). Finally, the stationary model was enhanced to a partly time-variable model. Applying the stationary and the time-variable models, we started a prediction well outside our calibration period, beginning with 100% bare soil in 1974 as the known start of the succession, and generated the coverage of 12 predefined vegetation types in three-year intervals. Transitions among vegetation types changed significantly in space and over time. While the probability of colonization was almost constant over time, the replacement rate tended to increase, indicating that the speed of succession accelerated with time or fluctuations became stronger. The predictions of both models agreed surprisingly well with the vegetation data observed more than two decades later. This shows that our dry grassland succession in a post-mining landscape can be adequately described by comparably simple types of Markov models, although some model assumptions have not been fulfilled and within-plot transitions have not been observed with point exactness. The major achievement of our proposed way to convert vegetation time series into transition matrices is the estimation of probability of events--a strength not provided by other frequently used statistical methods in vegetation science.

Indexed semi-Markov process for wind speed modeling.

NASA Astrophysics Data System (ADS)

Petroni, F.; D'Amico, G.; Prattico, F.

2012-04-01

The increasing interest in renewable energy leads scientific research to find a better way to recover most of the available energy. Particularly, the maximum energy recoverable from wind is equal to 59.3% of that available (Betz law) at a specific pitch angle and when the ratio between the wind speed in output and in input is equal to 1/3. The pitch angle is the angle formed between the airfoil of the blade of the wind turbine and the wind direction. Old turbine and a lot of that actually marketed, in fact, have always the same invariant geometry of the airfoil. This causes that wind turbines will work with an efficiency that is lower than 59.3%. New generation wind turbines, instead, have a system to variate the pitch angle by rotating the blades. This system able the wind turbines to recover, at different wind speed, always the maximum energy, working in Betz limit at different speed ratios. A powerful system control of the pitch angle allows the wind turbine to recover better the energy in transient regime. A good stochastic model for wind speed is then needed to help both the optimization of turbine design and to assist the system control to predict the value of the wind speed to positioning the blades quickly and correctly. The possibility to have synthetic data of wind speed is a powerful instrument to assist designer to verify the structures of the wind turbines or to estimate the energy recoverable from a specific site. To generate synthetic data, Markov chains of first or higher order are often used [1,2,3]. In particular in [1] is presented a comparison between a first-order Markov chain and a second-order Markov chain. A similar work, but only for the first-order Markov chain, is conduced by [2], presenting the probability transition matrix and comparing the energy spectral density and autocorrelation of real and synthetic wind speed data. A tentative to modeling and to join speed and direction of wind is presented in [3], by using two models, first-order Markov chain with different number of states, and Weibull distribution. All this model use Markov chains to generate synthetic wind speed time series but the search for a better model is still open. Approaching this issue, we applied new models which are generalization of Markov models. More precisely we applied semi-Markov models to generate synthetic wind speed time series. In a previous work we proposed different semi-Markov models, showing their ability to reproduce the autocorrelation structures of wind speed data. In that paper we showed also that the autocorrelation is higher with respect to the Markov model. Unfortunately this autocorrelation was still too small compared to the empirical one. In order to overcome the problem of low autocorrelation, in this paper we propose an indexed semi-Markov model. More precisely we assume that wind speed is described by a discrete time homogeneous semi-Markov process. We introduce a memory index which takes into account the periods of different wind activities. With this model the statistical characteristics of wind speed are faithfully reproduced. The wind is a very unstable phenomenon characterized by a sequence of lulls and sustained speeds, and a good wind generator must be able to reproduce such sequences. To check the validity of the predictive semi-Markovian model, the persistence of synthetic winds were calculated, then averaged and computed. The model is used to generate synthetic time series for wind speed by means of Monte Carlo simulations and the time lagged autocorrelation is used to compare statistical properties of the proposed models with those of real data and also with a time series generated though a simple Markov chain. [1] A. Shamshad, M.A. Bawadi, W.M.W. Wan Hussin, T.A. Majid, S.A.M. Sanusi, First and second order Markov chain models for synthetic generation of wind speed time series, Energy 30 (2005) 693-708. [2] H. Nfaoui, H. Essiarab, A.A.M. Sayigh, A stochastic Markov chain model for simulating wind speed time series at Tangiers, Morocco, Renewable Energy 29 (2004) 1407-1418. [3] F. Youcef Ettoumi, H. Sauvageot, A.-E.-H. Adane, Statistical bivariate modeling of wind using first-order Markov chain and Weibull distribution, Renewable Energy 28 (2003) 1787-1802.

Correlated gamma-based hidden Markov model for the smart asthma management based on rescue inhaler usage.

PubMed

Son, Junbo; Brennan, Patricia Flatley; Zhou, Shiyu

2017-05-10

Asthma is a very common chronic disease that affects a large portion of population in many nations. Driven by the fast development in sensor and mobile communication technology, a smart asthma management system has become available to continuously monitor the key health indicators of asthma patients. Such data provides opportunities for healthcare practitioners to examine patients not only in the clinic (on-site) but also outside of the clinic (off-site) in their daily life. In this paper, taking advantage from this data availability, we propose a correlated gamma-based hidden Markov model framework, which can reveal and highlight useful information from the rescue inhaler-usage profiles of individual patients for practitioners. The proposed method can provide diagnostic information about the asthma control status of individual patients and can help practitioners to make more informed therapeutic decisions accordingly. The proposed method is validated through both numerical study and case study based on real world data. Copyright © 2017 John Wiley & Sons, Ltd. Copyright © 2017 John Wiley & Sons, Ltd.

TOTAL user manual

NASA Technical Reports Server (NTRS)

Johnson, Sally C.; Boerschlein, David P.

1994-01-01

Semi-Markov models can be used to analyze the reliability of virtually any fault-tolerant system. However, the process of delineating all of the states and transitions in the model of a complex system can be devastatingly tedious and error-prone. Even with tools such as the Abstract Semi-Markov Specification Interface to the SURE Tool (ASSIST), the user must describe a system by specifying the rules governing the behavior of the system in order to generate the model. With the Table Oriented Translator to the ASSIST Language (TOTAL), the user can specify the components of a typical system and their attributes in the form of a table. The conditions that lead to system failure are also listed in a tabular form. The user can also abstractly specify dependencies with causes and effects. The level of information required is appropriate for system designers with little or no background in the details of reliability calculations. A menu-driven interface guides the user through the system description process, and the program updates the tables as new information is entered. The TOTAL program automatically generates an ASSIST input description to match the system description.

Optimizing Likelihood Models for Particle Trajectory Segmentation in Multi-State Systems.

PubMed

Young, Dylan Christopher; Scrimgeour, Jan

2018-06-19

Particle tracking offers significant insight into the molecular mechanics that govern the behav- ior of living cells. The analysis of molecular trajectories that transition between different motive states, such as diffusive, driven and tethered modes, is of considerable importance, with even single trajectories containing significant amounts of information about a molecule's environment and its interactions with cellular structures. Hidden Markov models (HMM) have been widely adopted to perform the segmentation of such complex tracks. In this paper, we show that extensive analysis of hidden Markov model outputs using data derived from multi-state Brownian dynamics simulations can be used both for the optimization of the likelihood models used to describe the states of the system and for characterization of the technique's failure mechanisms. This analysis was made pos- sible by the implementation of parallelized adaptive direct search algorithm on a Nvidia graphics processing unit. This approach provides critical information for the visualization of HMM failure and successful design of particle tracking experiments where trajectories contain multiple mobile states. © 2018 IOP Publishing Ltd.

Refining Markov state models for conformational dynamics using ensemble-averaged data and time-series trajectories

NASA Astrophysics Data System (ADS)

Matsunaga, Y.; Sugita, Y.

2018-06-01

A data-driven modeling scheme is proposed for conformational dynamics of biomolecules based on molecular dynamics (MD) simulations and experimental measurements. In this scheme, an initial Markov State Model (MSM) is constructed from MD simulation trajectories, and then, the MSM parameters are refined using experimental measurements through machine learning techniques. The second step can reduce the bias of MD simulation results due to inaccurate force-field parameters. Either time-series trajectories or ensemble-averaged data are available as a training data set in the scheme. Using a coarse-grained model of a dye-labeled polyproline-20, we compare the performance of machine learning estimations from the two types of training data sets. Machine learning from time-series data could provide the equilibrium populations of conformational states as well as their transition probabilities. It estimates hidden conformational states in more robust ways compared to that from ensemble-averaged data although there are limitations in estimating the transition probabilities between minor states. We discuss how to use the machine learning scheme for various experimental measurements including single-molecule time-series trajectories.

Upscaling of dilution and mixing using a trajectory based Spatial Markov random walk model in a periodic flow domain

NASA Astrophysics Data System (ADS)

Sund, Nicole L.; Porta, Giovanni M.; Bolster, Diogo

2017-05-01

The Spatial Markov Model (SMM) is an upscaled model that has been used successfully to predict effective mean transport across a broad range of hydrologic settings. Here we propose a novel variant of the SMM, applicable to spatially periodic systems. This SMM is built using particle trajectories, rather than travel times. By applying the proposed SMM to a simple benchmark problem we demonstrate that it can predict mean effective transport, when compared to data from fully resolved direct numerical simulations. Next we propose a methodology for using this SMM framework to predict measures of mixing and dilution, that do not just depend on mean concentrations, but are strongly impacted by pore-scale concentration fluctuations. We use information from trajectories of particles to downscale and reconstruct pore-scale approximate concentration fields from which mixing and dilution measures are then calculated. The comparison between measurements from fully resolved simulations and predictions with the SMM agree very favorably.

Link between unemployment and crime in the US: a Markov-Switching approach.

PubMed

Fallahi, Firouz; Rodríguez, Gabriel

2014-05-01

This study has two goals. The first is to use Markov Switching models to identify and analyze the cycles in the unemployment rate and four different types of property-related criminal activities in the US. The second is to apply the nonparametric concordance index of Harding and Pagan (2006) to determine the correlation between the cycles of unemployment rate and property crimes. Findings show that there is a positive but insignificant relationship between the unemployment rate, burglary, larceny, and robbery. However, the unemployment rate has a significant and negative (i.e., a counter-cyclical) relationship with motor-vehicle theft. Therefore, more motor-vehicle thefts occur during economic expansions relative to contractions. Next, we divide the sample into three different subsamples to examine the consistency of the findings. The results show that the co-movements between the unemployment rate and property crimes during recession periods are much weaker, when compared with that of the normal periods of the US economy. Copyright © 2013 Elsevier Inc. All rights reserved.

Invariant graphs of a family of non-uniformly expanding skew products over Markov maps

NASA Astrophysics Data System (ADS)

Walkden, C. P.; Withers, T.

2018-06-01

We consider a family of skew-products of the form where T is a continuous, expanding, locally eventually onto Markov map and is a family of homeomorphisms of . A function is said to be an invariant graph if is an invariant set for the skew-product; equivalently, u(T(x))  =  g x (u(x)). A well-studied problem is to consider the existence, regularity and dimension-theoretic properties of such functions, usually under strong contraction or expansion conditions (in terms of Lyapunov exponents or partial hyperbolicity) in the fibre direction. Here we consider such problems in a setting where the Lyapunov exponent in the fibre direction is zero on a set of periodic orbits but expands except on a neighbourhood of these periodic orbits. We prove that u either has the structure of a ‘quasi-graph’ (or ‘bony graph’) or is as smooth as the dynamics, and we give a criteria for this to happen.

Forecasting client transitions in British Columbia's Long-Term Care Program.

PubMed Central

Lane, D; Uyeno, D; Stark, A; Gutman, G; McCashin, B

1987-01-01

This article presents a model for the annual transitions of clients through various home and facility placements in a long-term care program. The model, an application of Markov chain analysis, is developed, tested, and applied to over 9,000 clients (N = 9,483) in British Columbia's Long Term Care Program (LTC) over the period 1978-1983. Results show that the model gives accurate forecasts of the progress of groups of clients from state to state in the long-term care system from time of admission until eventual death. Statistical methods are used to test the modeling hypothesis that clients' year-over-year transitions occur in constant proportions from state to state within the long-term care system. Tests are carried out by examining actual year-over-year transitions of each year's new admission cohort (1978-1983). Various subsets of the available data are analyzed and, after accounting for clear differences among annual cohorts, the most acceptable model of the actual client transition data occurred when clients were separated into male and female groups, i.e., the transition behavior of each group is describable by a different Markov model. To validate the model, we develop model estimates for the numbers of existing clients in each state of the long-term care system for the period (1981-1983) for which actual data are available. When these estimates are compared with the actual data, total weighted absolute deviations do not exceed 10 percent of actuals. Finally, we use the properties of the Markov chain probability transition matrix and simulation methods to develop three-year forecasts with prediction intervals for the distribution of the existing total clients into each state of the system. The tests, forecasts, and Markov model supplemental information are contained in a mechanized procedure suitable for a microcomputer. The procedure provides a powerful, efficient tool for decision makers planning facilities and services in response to the needs of long-term care clients. PMID:3121537

Revisiting Statistical Aspects of Nuclear Material Accounting

DOE PAGES

Burr, T.; Hamada, M. S.

2013-01-01

Nuclear material accounting (NMA) is the only safeguards system whose benefits are routinely quantified. Process monitoring (PM) is another safeguards system that is increasingly used, and one challenge is how to quantify its benefit. This paper considers PM in the role of enabling frequent NMA, which is referred to as near-real-time accounting (NRTA). We quantify NRTA benefits using period-driven and data-driven testing. Period-driven testing makes a decision to alarm or not at fixed periods. Data-driven testing decides as the data arrives whether to alarm or continue testing. The difference between period-driven and datad-riven viewpoints is illustrated by using one-year andmore » two-year periods. For both one-year and two-year periods, period-driven NMA using once-per-year cumulative material unaccounted for (CUMUF) testing is compared to more frequent Shewhart and joint sequential cusum testing using either MUF or standardized, independently transformed MUF (SITMUF) data. We show that the data-driven viewpoint is appropriate for NRTA and that it can be used to compare safeguards effectiveness. In addition to providing period-driven and data-driven viewpoints, new features include assessing the impact of uncertainty in the estimated covariance matrix of the MUF sequence and the impact of both random and systematic measurement errors.« less

Modeling and Properties of Nonlinear Stochastic Dynamical System of Continuous Culture

NASA Astrophysics Data System (ADS)

Wang, Lei; Feng, Enmin; Ye, Jianxiong; Xiu, Zhilong

The stochastic counterpart to the deterministic description of continuous fermentation with ordinary differential equation is investigated in the process of glycerol bio-dissimilation to 1,3-propanediol by Klebsiella pneumoniae. We briefly discuss the continuous fermentation process driven by three-dimensional Brownian motion and Lipschitz coefficients, which is suitable for the factual fermentation. Subsequently, we study the existence and uniqueness of solutions for the stochastic system as well as the boundedness of the Two-order Moment and the Markov property of the solution. Finally stochastic simulation is carried out under the Stochastic Euler-Maruyama method.

The application of Markov decision process in restaurant delivery robot

NASA Astrophysics Data System (ADS)

Wang, Yong; Hu, Zhen; Wang, Ying

2017-05-01

As the restaurant delivery robot is often in a dynamic and complex environment, including the chairs inadvertently moved to the channel and customers coming and going. The traditional path planning algorithm is not very ideal. To solve this problem, this paper proposes the Markov dynamic state immediate reward (MDR) path planning algorithm according to the traditional Markov decision process. First of all, it uses MDR to plan a global path, then navigates along this path. When the sensor detects there is no obstructions in front state, increase its immediate state reward value; when the sensor detects there is an obstacle in front, plan a global path that can avoid obstacle with the current position as the new starting point and reduce its state immediate reward value. This continues until the target is reached. When the robot learns for a period of time, it can avoid those places where obstacles are often present when planning the path. By analyzing the simulation experiment, the algorithm has achieved good results in the global path planning under the dynamic environment.

Extreme event statistics in a drifting Markov chain

NASA Astrophysics Data System (ADS)

Kindermann, Farina; Hohmann, Michael; Lausch, Tobias; Mayer, Daniel; Schmidt, Felix; Widera, Artur

2017-07-01

We analyze extreme event statistics of experimentally realized Markov chains with various drifts. Our Markov chains are individual trajectories of a single atom diffusing in a one-dimensional periodic potential. Based on more than 500 individual atomic traces we verify the applicability of the Sparre Andersen theorem to our system despite the presence of a drift. We present detailed analysis of four different rare-event statistics for our system: the distributions of extreme values, of record values, of extreme value occurrence in the chain, and of the number of records in the chain. We observe that, for our data, the shape of the extreme event distributions is dominated by the underlying exponential distance distribution extracted from the atomic traces. Furthermore, we find that even small drifts influence the statistics of extreme events and record values, which is supported by numerical simulations, and we identify cases in which the drift can be determined without information about the underlying random variable distributions. Our results facilitate the use of extreme event statistics as a signal for small drifts in correlated trajectories.

Using Markov Chains to predict the natural progression of diabetic retinopathy.

PubMed

Srikanth, Priyanka

2015-01-01

To study the natural progression of diabetic retinopathy in patients with type 2 diabetes. This was an observational study of 153 cases with type 2 diabetes from 2010 to 2013. The state of patient was noted at end of each year and transition matrices were developed to model movement between years. Patients who progressed to severe non-proliferative diabetic retinopathy (NPDR) were treated. Markov Chains and Chi-square test were used for statistical analysis. We modelled the transition of 153 patients from NPDR to blindness on an annual basis. At the end of year 3, we compared results from the Markov model versus actual data. The results from Chi-square test confirmed that there was statistically no significant difference (P=0.70) which provided assurance that the model was robust to estimate mean sojourn times. The key finding was that a patient entering the system in mild NPDR state is expected to stay in that state for 5y followed by 1.07y in moderate NPDR, be in the severe NPDR state for 1.33y before moving into PDR for roughly 8y. It is therefore expected that such a patient entering the model in a state of mild NPDR will enter blindness after 15.29y. Patients stay for long time periods in mild NPDR before transitioning into moderate NPDR. However, they move rapidly from moderate NPDR to proliferative diabetic retinopathy (PDR) and stay in that state for long periods before transitioning into blindness.

Dissipation, generalized free energy, and a self-consistent nonequilibrium thermodynamics of chemically driven open subsystems.

PubMed

Ge, Hao; Qian, Hong

2013-06-01

Nonequilibrium thermodynamics of a system situated in a sustained environment with influx and efflux is usually treated as a subsystem in a larger, closed "universe." A question remains with regard to what the minimally required description for the surrounding of such an open driven system is so that its nonequilibrium thermodynamics can be established solely based on the internal stochastic kinetics. We provide a solution to this problem using insights from studies of molecular motors in a chemical nonequilibrium steady state (NESS) with sustained external drive through a regenerating system or in a quasisteady state (QSS) with an excess amount of adenosine triphosphate (ATP), adenosine diphosphate (ADP), and inorganic phosphate (Pi). We introduce the key notion of minimal work that is needed, W(min), for the external regenerating system to sustain a NESS (e.g., maintaining constant concentrations of ATP, ADP and Pi for a molecular motor). Using a Markov (master-equation) description of a motor protein, we illustrate that the NESS and QSS have identical kinetics as well as the second law in terms of the same positive entropy production rate. The heat dissipation of a NESS without mechanical output is exactly the W(min). This provides a justification for introducing an ideal external regenerating system and yields a free-energy balance equation between the net free-energy input F(in) and total dissipation F(dis) in an NESS: F(in) consists of chemical input minus mechanical output; F(dis) consists of dissipative heat, i.e. the amount of useful energy becoming heat, which also equals the NESS entropy production. Furthermore, we show that for nonstationary systems, the F(dis) and F(in) correspond to the entropy production rate and housekeeping heat in stochastic thermodynamics and identify a relative entropy H as a generalized free energy. We reach a new formulation of Markovian nonequilibrium thermodynamics based on only the internal kinetic equation without further reference to the intrinsic degree of freedom within each Markov state. It includes an extended free-energy balance and a second law which are valid for driven stochastic dynamics with an ideal external regenerating system. Our result suggests new ingredients for a generalized thermodynamics of self-organization in driven systems.

Structured Set Intra Prediction With Discriminative Learning in a Max-Margin Markov Network for High Efficiency Video Coding

PubMed Central

Dai, Wenrui; Xiong, Hongkai; Jiang, Xiaoqian; Chen, Chang Wen

2014-01-01

This paper proposes a novel model on intra coding for High Efficiency Video Coding (HEVC), which simultaneously predicts blocks of pixels with optimal rate distortion. It utilizes the spatial statistical correlation for the optimal prediction based on 2-D contexts, in addition to formulating the data-driven structural interdependences to make the prediction error coherent with the probability distribution, which is desirable for successful transform and coding. The structured set prediction model incorporates a max-margin Markov network (M3N) to regulate and optimize multiple block predictions. The model parameters are learned by discriminating the actual pixel value from other possible estimates to maximize the margin (i.e., decision boundary bandwidth). Compared to existing methods that focus on minimizing prediction error, the M3N-based model adaptively maintains the coherence for a set of predictions. Specifically, the proposed model concurrently optimizes a set of predictions by associating the loss for individual blocks to the joint distribution of succeeding discrete cosine transform coefficients. When the sample size grows, the prediction error is asymptotically upper bounded by the training error under the decomposable loss function. As an internal step, we optimize the underlying Markov network structure to find states that achieve the maximal energy using expectation propagation. For validation, we integrate the proposed model into HEVC for optimal mode selection on rate-distortion optimization. The proposed prediction model obtains up to 2.85% bit rate reduction and achieves better visual quality in comparison to the HEVC intra coding. PMID:25505829

How old is this bird? The age distribution under some phase sampling schemes.

PubMed

Hautphenne, Sophie; Massaro, Melanie; Taylor, Peter

2017-12-01

In this paper, we use a finite-state continuous-time Markov chain with one absorbing state to model an individual's lifetime. Under this model, the time of death follows a phase-type distribution, and the transient states of the Markov chain are known as phases. We then attempt to provide an answer to the simple question "What is the conditional age distribution of the individual, given its current phase"? We show that the answer depends on how we interpret the question, and in particular, on the phase observation scheme under consideration. We then apply our results to the computation of the age pyramid for the endangered Chatham Island black robin Petroica traversi during the monitoring period 2007-2014.

Human Behavior Drift Detection in a Smart Home Environment.

PubMed

Masciadri, Andrea; Trofimova, Anna A; Matteucci, Matteo; Salice, Fabio

2017-01-01

The proposed system aims at elderly people independent living by providing an early indicator of habits changes which might be relevant for a diagnosis of diseases. It relies on Hidden Markov Model to describe the behavior observing sensors data, while Likelihood Ratio Test gives the variation within different time periods.

Reservoir optimisation using El Niño information. Case study of Daule Peripa (Ecuador)

NASA Astrophysics Data System (ADS)

Gelati, Emiliano; Madsen, Henrik; Rosbjerg, Dan

2010-05-01

The optimisation of water resources systems requires the ability to produce runoff scenarios that are consistent with available climatic information. We approach stochastic runoff modelling with a Markov-modulated autoregressive model with exogenous input, which belongs to the class of Markov-switching models. The model assumes runoff parameterisation to be conditioned on a hidden climatic state following a Markov chain, whose state transition probabilities depend on climatic information. This approach allows stochastic modeling of non-stationary runoff, as runoff anomalies are described by a mixture of autoregressive models with exogenous input, each one corresponding to a climate state. We calibrate the model on the inflows of the Daule Peripa reservoir located in western Ecuador, where the occurrence of El Niño leads to anomalously heavy rainfall caused by positive sea surface temperature anomalies along the coast. El Niño - Southern Oscillation (ENSO) information is used to condition the runoff parameterisation. Inflow predictions are realistic, especially at the occurrence of El Niño events. The Daule Peripa reservoir serves a hydropower plant and a downstream water supply facility. Using historical ENSO records, synthetic monthly inflow scenarios are generated for the period 1950-2007. These scenarios are used as input to perform stochastic optimisation of the reservoir rule curves with a multi-objective Genetic Algorithm (MOGA). The optimised rule curves are assumed to be the reservoir base policy. ENSO standard indices are currently forecasted at monthly time scale with nine-month lead time. These forecasts are used to perform stochastic optimisation of reservoir releases at each monthly time step according to the following procedure: (i) nine-month inflow forecast scenarios are generated using ENSO forecasts; (ii) a MOGA is set up to optimise the upcoming nine monthly releases; (iii) the optimisation is carried out by simulating the releases on the inflow forecasts, and by applying the base policy on a subsequent synthetic inflow scenario in order to account for long-term costs; (iv) the optimised release for the first month is implemented; (v) the state of the system is updated and (i), (ii), (iii), and (iv) are iterated for the following time step. The results highlight the advantages of using a climate-driven stochastic model to produce inflow scenarios and forecasts for reservoir optimisation, showing potential improvements with respect to the current management. Dynamic programming was used to find the best possible release time series given the inflow observations, in order to benchmark any possible operational improvement.

On Markov parameters in system identification

NASA Technical Reports Server (NTRS)

Phan, Minh; Juang, Jer-Nan; Longman, Richard W.

1991-01-01

A detailed discussion of Markov parameters in system identification is given. Different forms of input-output representation of linear discrete-time systems are reviewed and discussed. Interpretation of sampled response data as Markov parameters is presented. Relations between the state-space model and particular linear difference models via the Markov parameters are formulated. A generalization of Markov parameters to observer and Kalman filter Markov parameters for system identification is explained. These extended Markov parameters play an important role in providing not only a state-space realization, but also an observer/Kalman filter for the system of interest.

Nonlinear Markov Control Processes and Games

DTIC Science & Technology

2012-11-15

the analysis of a new class of stochastic games , nonlinear Markov games , as they arise as a ( competitive ) controlled version of nonlinear Markov... competitive interests) a nonlinear Markov game that we are investigating. I 0. :::tUt::JJt:.l.. I I t:t11VI;:, nonlinear Markov game , nonlinear Markov...corresponding stochastic game Γ+(T, h). In a slightly different setting one can assume that changes in a competitive control process occur as a

Zipf exponent of trajectory distribution in the hidden Markov model

NASA Astrophysics Data System (ADS)

Bochkarev, V. V.; Lerner, E. Yu

2014-03-01

This paper is the first step of generalization of the previously obtained full classification of the asymptotic behavior of the probability for Markov chain trajectories for the case of hidden Markov models. The main goal is to study the power (Zipf) and nonpower asymptotics of the frequency list of trajectories of hidden Markov frequencys and to obtain explicit formulae for the exponent of the power asymptotics. We consider several simple classes of hidden Markov models. We prove that the asymptotics for a hidden Markov model and for the corresponding Markov chain can be essentially different.

Using Markov Chains to predict the natural progression of diabetic retinopathy

PubMed Central

Srikanth, Priyanka

2015-01-01

AIM To study the natural progression of diabetic retinopathy in patients with type 2 diabetes. METHODS This was an observational study of 153 cases with type 2 diabetes from 2010 to 2013. The state of patient was noted at end of each year and transition matrices were developed to model movement between years. Patients who progressed to severe non-proliferative diabetic retinopathy (NPDR) were treated. Markov Chains and Chi-square test were used for statistical analysis. RESULTS We modelled the transition of 153 patients from NPDR to blindness on an annual basis. At the end of year 3, we compared results from the Markov model versus actual data. The results from Chi-square test confirmed that there was statistically no significant difference (P=0.70) which provided assurance that the model was robust to estimate mean sojourn times. The key finding was that a patient entering the system in mild NPDR state is expected to stay in that state for 5y followed by 1.07y in moderate NPDR, be in the severe NPDR state for 1.33y before moving into PDR for roughly 8y. It is therefore expected that such a patient entering the model in a state of mild NPDR will enter blindness after 15.29y. CONCLUSION Patients stay for long time periods in mild NPDR before transitioning into moderate NPDR. However, they move rapidly from moderate NPDR to proliferative diabetic retinopathy (PDR) and stay in that state for long periods before transitioning into blindness. PMID:25709923

Fluctuation theorems for discrete kinetic models of molecular motors

NASA Astrophysics Data System (ADS)

Faggionato, Alessandra; Silvestri, Vittoria

2017-04-01

Motivated by discrete kinetic models for non-cooperative molecular motors on periodic tracks, we consider random walks (also not Markov) on quasi one dimensional (1d) lattices, obtained by gluing several copies of a fundamental graph in a linear fashion. We show that, for a suitable class of quasi-1d lattices, the large deviation rate function associated to the position of the walker satisfies a Gallavotti-Cohen symmetry for any choice of the dynamical parameters defining the stochastic walk. This class includes the linear model considered in Lacoste et al (2008 Phys. Rev. E 78 011915). We also derive fluctuation theorems for the time-integrated cycle currents and discuss how the matrix approach of Lacoste et al (2008 Phys. Rev. E 78 011915) can be extended to derive the above Gallavotti-Cohen symmetry for any Markov random walk on {Z} with periodic jump rates. Finally, we review in the present context some large deviation results of Faggionato and Silvestri (2017 Ann. Inst. Henri Poincaré 53 46-78) and give some specific examples with explicit computations.

Exploring the WTI crude oil price bubble process using the Markov regime switching model

NASA Astrophysics Data System (ADS)

Zhang, Yue-Jun; Wang, Jing

2015-03-01

The sharp volatility of West Texas Intermediate (WTI) crude oil price in the past decade triggers us to investigate the price bubbles and their evolving process. Empirical results indicate that the fundamental price of WTI crude oil appears relatively more stable than that of the market-trading price, which verifies the existence of oil price bubbles during the sample period. Besides, by allowing the WTI crude oil price bubble process to switch between two states (regimes) according to a first-order Markov chain, we are able to statistically discriminate upheaval from stable states in the crude oil price bubble process; and in most of time, the stable state dominates the WTI crude oil price bubbles while the upheaval state usually proves short-lived and accompanies unexpected market events.

Quantum transport under ac drive from the leads: A Redfield quantum master equation approach

NASA Astrophysics Data System (ADS)

Purkayastha, Archak; Dubi, Yonatan

2017-08-01

Evaluating the time-dependent dynamics of driven open quantum systems is relevant for a theoretical description of many systems, including molecular junctions, quantum dots, cavity-QED experiments, cold atoms experiments, and more. Here, we formulate a rigorous microscopic theory of an out-of-equilibrium open quantum system of noninteracting particles on a lattice weakly coupled bilinearly to multiple baths and driven by periodically varying thermodynamic parameters like temperature and chemical potential of the bath. The particles can be either bosonic or fermionic and the lattice can be of any dimension and geometry. Based on the Redfield quantum master equation under Born-Markov approximation, we derive a linear differential equation for an equal time two point correlation matrix, sometimes also called a single-particle density matrix, from which various physical observables, for example, current, can be calculated. Various interesting physical effects, such as resonance, can be directly read off from the equations. Thus, our theory is quite general and gives quite transparent and easy-to-calculate results. We validate our theory by comparing with exact numerical simulations. We apply our method to a generic open quantum system, namely, a double quantum dot coupled to leads with modulating chemical potentials. The two most important experimentally relevant insights from this are as follows: (i) Time-dependent measurements of current for symmetric oscillating voltages (with zero instantaneous voltage bias) can point to the degree of asymmetry in the system-bath coupling and (ii) under certain conditions time-dependent currents can exceed time-averaged currents by several orders of magnitude, and can therefore be detected even when the average current is below the measurement threshold.

Modeling Land Use/Cover Changes in an African Rural Landscape

NASA Astrophysics Data System (ADS)

Kamusoko, C.; Aniya, M.

2006-12-01

Land use/cover changes are analyzed in the Bindura district of Zimbabwe, Africa through the integration of data from a time series of Landsat imagery (1973, 1989 and 2000), a household survey and GIS coverages. We employed a hybrid supervised/unsupervised classification approach to generate land use/cover maps from which landscape metrics were calculated. Population and other household variables were derived from a sample of surveyed villages, while road accessibility and slope were obtained from topographic maps and digital elevation model, respectively. Markov-cellular automata modeling approach that incorporates Markov chain analysis, cellular automata and multi-criteria evaluation (MCE) / multi-objective allocation (MOLA) procedures was used to simulate land use/cover changes. A GIS-based MCE technique computed transition potential maps, whereas transition areas were derived from the 1973-2000 land use/cover maps using the Markov chain analysis. A 5 x 5 cellular automata filter was used to develop a spatially explicit contiguity- weighting factor to change the cells based on its previous state and those of its neighbors, while MOLA resolved land use/cover class allocation conflicts. The kappa index of agreement was used for model validation. Observed trends in land use/cover changes indicate that deforestation and the encroachment of cultivation in woodland areas is a continuous trend in the study area. This suggests that economic activities driven by agricultural expansion were the main causes of landscape fragmentation, leading to landscape degradation. Rigorous calibration of transition potential maps done by a MCE algorithm and Markovian transition probabilities produced accurate inputs for the simulation of land use/cover changes. Overall standard kappa index of agreement ranged from 0.73 to 0.83, which is sufficient for simulating land use/cover changes in the study area. Land use/cover simulations under the 1989 and 2000 scenario indicated further landscape degradation in the rural areas of the Bindura district. Keywords: Zimbabwe, land use/cover changes, landscape fragmentation, GIS, land use/cover change modeling, multi-criteria evaluation/multi-objective allocation procedures, Markov-cellular automata

An Overview of Markov Chain Methods for the Study of Stage-Sequential Developmental Processes

ERIC Educational Resources Information Center

Kapland, David

2008-01-01

This article presents an overview of quantitative methodologies for the study of stage-sequential development based on extensions of Markov chain modeling. Four methods are presented that exemplify the flexibility of this approach: the manifest Markov model, the latent Markov model, latent transition analysis, and the mixture latent Markov model.…

A novel framework to simulating non-stationary, non-linear, non-Normal hydrological time series using Markov Switching Autoregressive Models

NASA Astrophysics Data System (ADS)

Birkel, C.; Paroli, R.; Spezia, L.; Tetzlaff, D.; Soulsby, C.

2012-12-01

In this paper we present a novel model framework using the class of Markov Switching Autoregressive Models (MSARMs) to examine catchments as complex stochastic systems that exhibit non-stationary, non-linear and non-Normal rainfall-runoff and solute dynamics. Hereby, MSARMs are pairs of stochastic processes, one observed and one unobserved, or hidden. We model the unobserved process as a finite state Markov chain and assume that the observed process, given the hidden Markov chain, is conditionally autoregressive, which means that the current observation depends on its recent past (system memory). The model is fully embedded in a Bayesian analysis based on Markov Chain Monte Carlo (MCMC) algorithms for model selection and uncertainty assessment. Hereby, the autoregressive order and the dimension of the hidden Markov chain state-space are essentially self-selected. The hidden states of the Markov chain represent unobserved levels of variability in the observed process that may result from complex interactions of hydroclimatic variability on the one hand and catchment characteristics affecting water and solute storage on the other. To deal with non-stationarity, additional meteorological and hydrological time series along with a periodic component can be included in the MSARMs as covariates. This extension allows identification of potential underlying drivers of temporal rainfall-runoff and solute dynamics. We applied the MSAR model framework to streamflow and conservative tracer (deuterium and oxygen-18) time series from an intensively monitored 2.3 km2 experimental catchment in eastern Scotland. Statistical time series analysis, in the form of MSARMs, suggested that the streamflow and isotope tracer time series are not controlled by simple linear rules. MSARMs showed that the dependence of current observations on past inputs observed by transport models often in form of the long-tailing of travel time and residence time distributions can be efficiently explained by non-stationarity either of the system input (climatic variability) and/or the complexity of catchment storage characteristics. The statistical model is also capable of reproducing short (event) and longer-term (inter-event) and wet and dry dynamical "hydrological states". These reflect the non-linear transport mechanisms of flow pathways induced by transient climatic and hydrological variables and modified by catchment characteristics. We conclude that MSARMs are a powerful tool to analyze the temporal dynamics of hydrological data, allowing for explicit integration of non-stationary, non-linear and non-Normal characteristics.

Markov stochasticity coordinates

DOE Office of Scientific and Technical Information (OSTI.GOV)

Eliazar, Iddo, E-mail: iddo.eliazar@intel.com

Markov dynamics constitute one of the most fundamental models of random motion between the states of a system of interest. Markov dynamics have diverse applications in many fields of science and engineering, and are particularly applicable in the context of random motion in networks. In this paper we present a two-dimensional gauging method of the randomness of Markov dynamics. The method–termed Markov Stochasticity Coordinates–is established, discussed, and exemplified. Also, the method is tweaked to quantify the stochasticity of the first-passage-times of Markov dynamics, and the socioeconomic equality and mobility in human societies.

An Efficient MCMC Algorithm to Sample Binary Matrices with Fixed Marginals

ERIC Educational Resources Information Center

Verhelst, Norman D.

2008-01-01

Uniform sampling of binary matrices with fixed margins is known as a difficult problem. Two classes of algorithms to sample from a distribution not too different from the uniform are studied in the literature: importance sampling and Markov chain Monte Carlo (MCMC). Existing MCMC algorithms converge slowly, require a long burn-in period and yield…

A compositional framework for Markov processes

NASA Astrophysics Data System (ADS)

Baez, John C.; Fong, Brendan; Pollard, Blake S.

2016-03-01

We define the concept of an "open" Markov process, or more precisely, continuous-time Markov chain, which is one where probability can flow in or out of certain states called "inputs" and "outputs." One can build up a Markov process from smaller open pieces. This process is formalized by making open Markov processes into the morphisms of a dagger compact category. We show that the behavior of a detailed balanced open Markov process is determined by a principle of minimum dissipation, closely related to Prigogine's principle of minimum entropy production. Using this fact, we set up a functor mapping open detailed balanced Markov processes to open circuits made of linear resistors. We also describe how to "black box" an open Markov process, obtaining the linear relation between input and output data that holds in any steady state, including nonequilibrium steady states with a nonzero flow of probability through the system. We prove that black boxing gives a symmetric monoidal dagger functor sending open detailed balanced Markov processes to Lagrangian relations between symplectic vector spaces. This allows us to compute the steady state behavior of an open detailed balanced Markov process from the behaviors of smaller pieces from which it is built. We relate this black box functor to a previously constructed black box functor for circuits.

Land-use change may exacerbate climate change impacts on water resources in the Ganges basin

NASA Astrophysics Data System (ADS)

Tsarouchi, Gina; Buytaert, Wouter

2018-02-01

Quantifying how land-use change and climate change affect water resources is a challenge in hydrological science. This work aims to quantify how future projections of land-use and climate change might affect the hydrological response of the Upper Ganges river basin in northern India, which experiences monsoon flooding almost every year. Three different sets of modelling experiments were run using the Joint UK Land Environment Simulator (JULES) land surface model (LSM) and covering the period 2000-2035: in the first set, only climate change is taken into account, and JULES was driven by the CMIP5 (Coupled Model Intercomparison Project Phase 5) outputs of 21 models, under two representative concentration pathways (RCP4.5 and RCP8.5), whilst land use was held fixed at the year 2010. In the second set, only land-use change is taken into account, and JULES was driven by a time series of 15 future land-use pathways, based on Landsat satellite imagery and the Markov chain simulation, whilst the meteorological boundary conditions were held fixed at years 2000-2005. In the third set, both climate change and land-use change were taken into consideration, as the CMIP5 model outputs were used in conjunction with the 15 future land-use pathways to force JULES. Variations in hydrological variables (stream flow, evapotranspiration and soil moisture) are calculated during the simulation period. Significant changes in the near-future (years 2030-2035) hydrologic fluxes arise under future land-cover and climate change scenarios pointing towards a severe increase in high extremes of flow: the multi-model mean of the 95th percentile of streamflow (Q5) is projected to increase by 63 % under the combined land-use and climate change high emissions scenario (RCP8.5). The changes in all examined hydrological components are greater in the combined land-use and climate change experiment. Results are further presented in a water resources context, aiming to address potential implications of climate change and land-use change from a water demand perspective. We conclude that future water demands in the Upper Ganges region for winter months may not be met.

No Magic Bullet: A Theory-Based Meta-Analysis of Markov Transition Probabilities in Studies of Service Systems for Persons With Mental Disabilities.

PubMed

Leff, Hugh Stephen; Chow, Clifton M; Graves, Stephen C

2017-03-01

A random-effects meta-analysis of studies that used Markov transition probabilities (TPs) to describe outcomes for mental health service systems of differing quality for persons with serious mental illness was implemented to improve the scientific understanding of systems performance, to use in planning simulations to project service system costs and outcomes over time, and to test a theory of how outcomes for systems varying in quality differ. Nineteen systems described in 12 studies were coded as basic (B), maintenance (M), and recovery oriented (R) on the basis of descriptions of services provided. TPs for studies were aligned with a common functional-level framework, converted to a one-month time period, synthesized, and compared with theory-based expectations. Meta-regression was employed to explore associations between TPs and characteristics of service recipients and studies. R systems performed better than M and B systems. However, M systems did not perform better than B systems. All systems showed negative as well as positive TPs. For approximately one-third of synthesized TPs, substantial interstudy heterogeneity was noted. Associations were found between TPs and service recipient and study variables Conclusions: Conceptualizing systems as B, M, and R has potential for improving scientific understanding and systems planning. R systems appear more effective than B and M systems, although there is no "magic bullet" system for all service recipients. Interstudy heterogeneity indicates need for common approaches to reporting service recipient states, time periods for TPs, service recipient attributes, and service system characteristics. TPs found should be used in Markov simulations to project system effectiveness and costs of over time.

Markov Model of Accident Progression at Fukushima Daiichi

DOE Office of Scientific and Technical Information (OSTI.GOV)

Cuadra A.; Bari R.; Cheng, L-Y

2012-11-11

On March 11, 2011, a magnitude 9.0 earthquake followed by a tsunami caused loss of offsite power and disabled the emergency diesel generators, leading to a prolonged station blackout at the Fukushima Daiichi site. After successful reactor trip for all operating reactors, the inability to remove decay heat over an extended period led to boil-off of the water inventory and fuel uncovery in Units 1-3. A significant amount of metal-water reaction occurred, as evidenced by the quantities of hydrogen generated that led to hydrogen explosions in the auxiliary buildings of the Units 1 & 3, and in the de-fuelled Unitmore » 4. Although it was assumed that extensive fuel damage, including fuel melting, slumping, and relocation was likely to have occurred in the core of the affected reactors, the status of the fuel, vessel, and drywell was uncertain. To understand the possible evolution of the accident conditions at Fukushima Daiichi, a Markov model of the likely state of one of the reactors was constructed and executed under different assumptions regarding system performance and reliability. The Markov approach was selected for several reasons: It is a probabilistic model that provides flexibility in scenario construction and incorporates time dependence of different model states. It also readily allows for sensitivity and uncertainty analyses of different failure and repair rates of cooling systems. While the analysis was motivated by a need to gain insight on the course of events for the damaged units at Fukushima Daiichi, the work reported here provides a more general analytical basis for studying and evaluating severe accident evolution over extended periods of time. This work was performed at the request of the U.S. Department of Energy to explore 'what-if' scenarios in the immediate aftermath of the accidents.« less

Multiscale climate emulator of multimodal wave spectra: MUSCLE-spectra

NASA Astrophysics Data System (ADS)

Rueda, Ana; Hegermiller, Christie A.; Antolinez, Jose A. A.; Camus, Paula; Vitousek, Sean; Ruggiero, Peter; Barnard, Patrick L.; Erikson, Li H.; Tomás, Antonio; Mendez, Fernando J.

2017-02-01

Characterization of multimodal directional wave spectra is important for many offshore and coastal applications, such as marine forecasting, coastal hazard assessment, and design of offshore wave energy farms and coastal structures. However, the multivariate and multiscale nature of wave climate variability makes this complex problem tractable using computationally expensive numerical models. So far, the skill of statistical-downscaling model-based parametric (unimodal) wave conditions is limited in large ocean basins such as the Pacific. The recent availability of long-term directional spectral data from buoys and wave hindcast models allows for development of stochastic models that include multimodal sea-state parameters. This work introduces a statistical downscaling framework based on weather types to predict multimodal wave spectra (e.g., significant wave height, mean wave period, and mean wave direction from different storm systems, including sea and swells) from large-scale atmospheric pressure fields. For each weather type, variables of interest are modeled using the categorical distribution for the sea-state type, the Generalized Extreme Value (GEV) distribution for wave height and wave period, a multivariate Gaussian copula for the interdependence between variables, and a Markov chain model for the chronology of daily weather types. We apply the model to the southern California coast, where local seas and swells from both the Northern and Southern Hemispheres contribute to the multimodal wave spectrum. This work allows attribution of particular extreme multimodal wave events to specific atmospheric conditions, expanding knowledge of time-dependent, climate-driven offshore and coastal sea-state conditions that have a significant influence on local nearshore processes, coastal morphology, and flood hazards.

Multiscale Climate Emulator of Multimodal Wave Spectra: MUSCLE-spectra

NASA Astrophysics Data System (ADS)

Rueda, A.; Hegermiller, C.; Alvarez Antolinez, J. A.; Camus, P.; Vitousek, S.; Ruggiero, P.; Barnard, P.; Erikson, L. H.; Tomas, A.; Mendez, F. J.

2016-12-01

Characterization of multimodal directional wave spectra is important for many offshore and coastal applications, such as marine forecasting, coastal hazard assessment, and design of offshore wave energy farms and coastal structures. However, the multivariate and multiscale nature of wave climate variability makes this problem complex yet tractable using computationally-expensive numerical models. So far, the skill of statistical-downscaling models based parametric (unimodal) wave conditions is limited in large ocean basins such as the Pacific. The recent availability of long-term directional spectral data from buoys and wave hindcast models allows for development of stochastic models that include multimodal sea-state parameters. This work introduces a statistical-downscaling framework based on weather types to predict multimodal wave spectra (e.g., significant wave height, mean wave period, and mean wave direction from different storm systems, including sea and swells) from large-scale atmospheric pressure fields. For each weather type, variables of interest are modeled using the categorical distribution for the sea-state type, the Generalized Extreme Value (GEV) distribution for wave height and wave period, a multivariate Gaussian copula for the interdependence between variables, and a Markov chain model for the chronology of daily weather types. We apply the model to the Southern California coast, where local seas and swells from both the Northern and Southern Hemispheres contribute to the multimodal wave spectrum. This work allows attribution of particular extreme multimodal wave events to specific atmospheric conditions, expanding knowledge of time-dependent, climate-driven offshore and coastal sea-state conditions that have a significant influence on local nearshore processes, coastal morphology, and flood hazards.

Model‐Based Approach to Predict Adherence to Protocol During Antiobesity Trials

PubMed Central

Sharma, Vishnu D.; Combes, François P.; Vakilynejad, Majid; Lahu, Gezim; Lesko, Lawrence J.

2017-01-01

Abstract Development of antiobesity drugs is continuously challenged by high dropout rates during clinical trials. The objective was to develop a population pharmacodynamic model that describes the temporal changes in body weight, considering disease progression, lifestyle intervention, and drug effects. Markov modeling (MM) was applied for quantification and characterization of responder and nonresponder as key drivers of dropout rates, to ultimately support the clinical trial simulations and the outcome in terms of trial adherence. Subjects (n = 4591) from 6 Contrave® trials were included in this analysis. An indirect‐response model developed by van Wart et al was used as a starting point. Inclusion of drug effect was dose driven using a population dose‐ and time‐dependent pharmacodynamic (DTPD) model. Additionally, a population‐pharmacokinetic parameter‐ and data (PPPD)‐driven model was developed using the final DTPD model structure and final parameter estimates from a previously developed population pharmacokinetic model based on available Contrave® pharmacokinetic concentrations. Last, MM was developed to predict transition rate probabilities among responder, nonresponder, and dropout states driven by the pharmacodynamic effect resulting from the DTPD or PPPD model. Covariates included in the models and parameters were diabetes mellitus and race. The linked DTPD‐MM and PPPD‐MM was able to predict transition rates among responder, nonresponder, and dropout states well. The analysis concluded that body‐weight change is an important factor influencing dropout rates, and the MM depicted that overall a DTPD model‐driven approach provides a reasonable prediction of clinical trial outcome probabilities similar to a pharmacokinetic‐driven approach. PMID:28858397

A Markov chain model for image ranking system in social networks

NASA Astrophysics Data System (ADS)

Zin, Thi Thi; Tin, Pyke; Toriu, Takashi; Hama, Hiromitsu

2014-03-01

In today world, different kinds of networks such as social, technological, business and etc. exist. All of the networks are similar in terms of distributions, continuously growing and expanding in large scale. Among them, many social networks such as Facebook, Twitter, Flickr and many others provides a powerful abstraction of the structure and dynamics of diverse kinds of inter personal connection and interaction. Generally, the social network contents are created and consumed by the influences of all different social navigation paths that lead to the contents. Therefore, identifying important and user relevant refined structures such as visual information or communities become major factors in modern decision making world. Moreover, the traditional method of information ranking systems cannot be successful due to their lack of taking into account the properties of navigation paths driven by social connections. In this paper, we propose a novel image ranking system in social networks by using the social data relational graphs from social media platform jointly with visual data to improve the relevance between returned images and user intentions (i.e., social relevance). Specifically, we propose a Markov chain based Social-Visual Ranking algorithm by taking social relevance into account. By using some extensive experiments, we demonstrated the significant and effectiveness of the proposed social-visual ranking method.

Hyper-Spectral Image Analysis With Partially Latent Regression and Spatial Markov Dependencies

NASA Astrophysics Data System (ADS)

Deleforge, Antoine; Forbes, Florence; Ba, Sileye; Horaud, Radu

2015-09-01

Hyper-spectral data can be analyzed to recover physical properties at large planetary scales. This involves resolving inverse problems which can be addressed within machine learning, with the advantage that, once a relationship between physical parameters and spectra has been established in a data-driven fashion, the learned relationship can be used to estimate physical parameters for new hyper-spectral observations. Within this framework, we propose a spatially-constrained and partially-latent regression method which maps high-dimensional inputs (hyper-spectral images) onto low-dimensional responses (physical parameters such as the local chemical composition of the soil). The proposed regression model comprises two key features. Firstly, it combines a Gaussian mixture of locally-linear mappings (GLLiM) with a partially-latent response model. While the former makes high-dimensional regression tractable, the latter enables to deal with physical parameters that cannot be observed or, more generally, with data contaminated by experimental artifacts that cannot be explained with noise models. Secondly, spatial constraints are introduced in the model through a Markov random field (MRF) prior which provides a spatial structure to the Gaussian-mixture hidden variables. Experiments conducted on a database composed of remotely sensed observations collected from the Mars planet by the Mars Express orbiter demonstrate the effectiveness of the proposed model.

Semi-Markov adjunction to the Computer-Aided Markov Evaluator (CAME)

NASA Technical Reports Server (NTRS)

Rosch, Gene; Hutchins, Monica A.; Leong, Frank J.; Babcock, Philip S., IV

1988-01-01

The rule-based Computer-Aided Markov Evaluator (CAME) program was expanded in its ability to incorporate the effect of fault-handling processes into the construction of a reliability model. The fault-handling processes are modeled as semi-Markov events and CAME constructs and appropriate semi-Markov model. To solve the model, the program outputs it in a form which can be directly solved with the Semi-Markov Unreliability Range Evaluator (SURE) program. As a means of evaluating the alterations made to the CAME program, the program is used to model the reliability of portions of the Integrated Airframe/Propulsion Control System Architecture (IAPSA 2) reference configuration. The reliability predictions are compared with a previous analysis. The results bear out the feasibility of utilizing CAME to generate appropriate semi-Markov models to model fault-handling processes.

Dynamic Latent Trait Models with Mixed Hidden Markov Structure for Mixed Longitudinal Outcomes.

PubMed

Zhang, Yue; Berhane, Kiros

2016-01-01

We propose a general Bayesian joint modeling approach to model mixed longitudinal outcomes from the exponential family for taking into account any differential misclassification that may exist among categorical outcomes. Under this framework, outcomes observed without measurement error are related to latent trait variables through generalized linear mixed effect models. The misclassified outcomes are related to the latent class variables, which represent unobserved real states, using mixed hidden Markov models (MHMM). In addition to enabling the estimation of parameters in prevalence, transition and misclassification probabilities, MHMMs capture cluster level heterogeneity. A transition modeling structure allows the latent trait and latent class variables to depend on observed predictors at the same time period and also on latent trait and latent class variables at previous time periods for each individual. Simulation studies are conducted to make comparisons with traditional models in order to illustrate the gains from the proposed approach. The new approach is applied to data from the Southern California Children Health Study (CHS) to jointly model questionnaire based asthma state and multiple lung function measurements in order to gain better insight about the underlying biological mechanism that governs the inter-relationship between asthma state and lung function development.

Patchwork sampling of stochastic differential equations

NASA Astrophysics Data System (ADS)

Kürsten, Rüdiger; Behn, Ulrich

2016-03-01

We propose a method to sample stationary properties of solutions of stochastic differential equations, which is accurate and efficient if there are rarely visited regions or rare transitions between distinct regions of the state space. The method is based on a complete, nonoverlapping partition of the state space into patches on which the stochastic process is ergodic. On each of these patches we run simulations of the process strictly truncated to the corresponding patch, which allows effective simulations also in rarely visited regions. The correct weight for each patch is obtained by counting the attempted transitions between all different patches. The results are patchworked to cover the whole state space. We extend the concept of truncated Markov chains which is originally formulated for processes which obey detailed balance to processes not fulfilling detailed balance. The method is illustrated by three examples, describing the one-dimensional diffusion of an overdamped particle in a double-well potential, a system of many globally coupled overdamped particles in double-well potentials subject to additive Gaussian white noise, and the overdamped motion of a particle on the circle in a periodic potential subject to a deterministic drift and additive noise. In an appendix we explain how other well-known Markov chain Monte Carlo algorithms can be related to truncated Markov chains.

Modelling past land use using archaeological and pollen data

NASA Astrophysics Data System (ADS)

Pirzamanbein, Behnaz; Lindström, johan; Poska, Anneli; Gaillard-Lemdahl, Marie-José

2016-04-01

Accurate maps of past land use are necessary for studying the impact of anthropogenic land-cover changes on climate and biodiversity. We develop a Bayesian hierarchical model to reconstruct the land use using Gaussian Markov random fields. The model uses two observations sets: 1) archaeological data, representing human settlements, urbanization and agricultural findings; and 2) pollen-based land estimates of the three land-cover types Coniferous forest, Broadleaved forest and Unforested/Open land. The pollen based estimates are obtained from the REVEALS model, based on pollen counts from lakes and bogs. Our developed model uses the sparse pollen-based estimations to reconstruct the spatial continuous cover of three land cover types. Using the open-land component and the archaeological data, the extent of land-use is reconstructed. The model is applied on three time periods - centred around 1900 CE, 1000 and, 4000 BCE over Sweden for which both pollen-based estimates and archaeological data are available. To estimate the model parameters and land use, a block updated Markov chain Monte Carlo (MCMC) algorithm is applied. Using the MCMC posterior samples uncertainties in land-use predictions are computed. Due to lack of good historic land use data, model results are evaluated by cross-validation. Keywords. Spatial reconstruction, Gaussian Markov random field, Fossil pollen records, Archaeological data, Human land-use, Prediction uncertainty

Markov-chain model of classified atomistic transition states for discrete kinetic Monte Carlo simulations.

PubMed

Numazawa, Satoshi; Smith, Roger

2011-10-01

Classical harmonic transition state theory is considered and applied in discrete lattice cells with hierarchical transition levels. The scheme is then used to determine transitions that can be applied in a lattice-based kinetic Monte Carlo (KMC) atomistic simulation model. The model results in an effective reduction of KMC simulation steps by utilizing a classification scheme of transition levels for thermally activated atomistic diffusion processes. Thermally activated atomistic movements are considered as local transition events constrained in potential energy wells over certain local time periods. These processes are represented by Markov chains of multidimensional Boolean valued functions in three-dimensional lattice space. The events inhibited by the barriers under a certain level are regarded as thermal fluctuations of the canonical ensemble and accepted freely. Consequently, the fluctuating system evolution process is implemented as a Markov chain of equivalence class objects. It is shown that the process can be characterized by the acceptance of metastable local transitions. The method is applied to a problem of Au and Ag cluster growth on a rippled surface. The simulation predicts the existence of a morphology-dependent transition time limit from a local metastable to stable state for subsequent cluster growth by accretion. Excellent agreement with observed experimental results is obtained.

Connected word recognition using a cascaded neuro-computational model

NASA Astrophysics Data System (ADS)

Hoya, Tetsuya; van Leeuwen, Cees

2016-10-01

We propose a novel framework for processing a continuous speech stream that contains a varying number of words, as well as non-speech periods. Speech samples are segmented into word-tokens and non-speech periods. An augmented version of an earlier-proposed, cascaded neuro-computational model is used for recognising individual words within the stream. Simulation studies using both a multi-speaker-dependent and speaker-independent digit string database show that the proposed method yields a recognition performance comparable to that obtained by a benchmark approach using hidden Markov models with embedded training.

Open Markov Processes and Reaction Networks

ERIC Educational Resources Information Center

Swistock Pollard, Blake Stephen

2017-01-01

We begin by defining the concept of "open" Markov processes, which are continuous-time Markov chains where probability can flow in and out through certain "boundary" states. We study open Markov processes which in the absence of such boundary flows admit equilibrium states satisfying detailed balance, meaning that the net flow…

The explicit form of the rate function for semi-Markov processes and its contractions

NASA Astrophysics Data System (ADS)

Sughiyama, Yuki; Kobayashi, Testuya J.

2018-03-01

We derive the explicit form of the rate function for semi-Markov processes. Here, the ‘random time change trick’ plays an essential role. Also, by exploiting the contraction principle of large deviation theory to the explicit form, we show that the fluctuation theorem (Gallavotti-Cohen symmetry) holds for semi-Markov cases. Furthermore, we elucidate that our rate function is an extension of the level 2.5 rate function for Markov processes to semi-Markov cases.

Communication: Introducing prescribed biases in out-of-equilibrium Markov models

NASA Astrophysics Data System (ADS)

Dixit, Purushottam D.

2018-03-01

Markov models are often used in modeling complex out-of-equilibrium chemical and biochemical systems. However, many times their predictions do not agree with experiments. We need a systematic framework to update existing Markov models to make them consistent with constraints that are derived from experiments. Here, we present a framework based on the principle of maximum relative path entropy (minimum Kullback-Leibler divergence) to update Markov models using stationary state and dynamical trajectory-based constraints. We illustrate the framework using a biochemical model network of growth factor-based signaling. We also show how to find the closest detailed balanced Markov model to a given Markov model. Further applications and generalizations are discussed.

Non-Linear Dynamics of Saturn’s Rings

NASA Astrophysics Data System (ADS)

Esposito, Larry W.

2015-11-01

Non-linear processes can explain why Saturn’s rings are so active and dynamic. Ring systems differ from simple linear systems in two significant ways: 1. They are systems of granular material: where particle-to-particle collisions dominate; thus a kinetic, not a fluid description needed. We find that stresses are strikingly inhomogeneous and fluctuations are large compared to equilibrium. 2. They are strongly forced by resonances: which drive a non-linear response, pushing the system across thresholds that lead to persistent states.Some of this non-linearity is captured in a simple Predator-Prey Model: Periodic forcing from the moon causes streamline crowding; This damps the relative velocity, and allows aggregates to grow. About a quarter phase later, the aggregates stir the system to higher relative velocity and the limit cycle repeats each orbit.Summary of Halo Results: A predator-prey model for ring dynamics produces transient structures like ‘straw’ that can explain the halo structure and spectroscopy: This requires energetic collisions (v ≈ 10m/sec, with throw distances about 200km, implying objects of scale R ≈ 20km).Transform to Duffing Eqn : With the coordinate transformation, z = M2/3, the Predator-Prey equations can be combined to form a single second-order differential equation with harmonic resonance forcing.Ring dynamics and history implications: Moon-triggered clumping at perturbed regions in Saturn’s rings creates both high velocity dispersion and large aggregates at these distances, explaining both small and large particles observed there. We calculate the stationary size distribution using a cell-to-cell mapping procedure that converts the phase-plane trajectories to a Markov chain. Approximating the Markov chain as an asymmetric random walk with reflecting boundaries allows us to determine the power law index from results of numerical simulations in the tidal environment surrounding Saturn. Aggregates can explain many dynamic aspects of the rings and can renew rings by shielding and recycling the material within them, depending on how long the mass is sequestered. We can ask: Are Saturn’s rings a chaotic non-linear driven system?

Non-Linear Dynamics of Saturn's Rings

NASA Astrophysics Data System (ADS)

Esposito, L. W.

2015-12-01

Non-linear processes can explain why Saturn's rings are so active and dynamic. Some of this non-linearity is captured in a simple Predator-Prey Model: Periodic forcing from the moon causes streamline crowding; This damps the relative velocity, and allows aggregates to grow. About a quarter phase later, the aggregates stir the system to higher relative velocity and the limit cycle repeats each orbit, with relative velocity ranging from nearly zero to a multiple of the orbit average: 2-10x is possible. Summary of Halo Results: A predator-prey model for ring dynamics produces transient structures like 'straw' that can explain the halo structure and spectroscopy: Cyclic velocity changes cause perturbed regions to reach higher collision speeds at some orbital phases, which preferentially removes small regolith particles; Surrounding particles diffuse back too slowly to erase the effect: this gives the halo morphology; This requires energetic collisions (v ≈ 10m/sec, with throw distances about 200km, implying objects of scale R ≈ 20km); We propose 'straw', as observed ny Cassini cameras. Transform to Duffing Eqn : With the coordinate transformation, z = M2/3, the Predator-Prey equations can be combined to form a single second-order differential equation with harmonic resonance forcing. Ring dynamics and history implications: Moon-triggered clumping at perturbed regions in Saturn's rings creates both high velocity dispersion and large aggregates at these distances, explaining both small and large particles observed there. This confirms the triple architecture of ring particles: a broad size distribution of particles; these aggregate into temporary rubble piles; coated by a regolith of dust. We calculate the stationary size distribution using a cell-to-cell mapping procedure that converts the phase-plane trajectories to a Markov chain. Approximating the Markov chain as an asymmetric random walk with reflecting boundaries allows us to determine the power law index from results of numerical simulations in the tidal environment surrounding Saturn. Aggregates can explain many dynamic aspects of the rings and can renew rings by shielding and recycling the material within them, depending on how long the mass is sequestered. We can ask: Are Saturn's rings a chaotic non-linear driven system?

Poissonian steady states: from stationary densities to stationary intensities.

PubMed

Eliazar, Iddo

2012-10-01

Markov dynamics are the most elemental and omnipresent form of stochastic dynamics in the sciences, with applications ranging from physics to chemistry, from biology to evolution, and from economics to finance. Markov dynamics can be either stationary or nonstationary. Stationary Markov dynamics represent statistical steady states and are quantified by stationary densities. In this paper, we generalize the notion of steady state to the case of general Markov dynamics. Considering an ensemble of independent motions governed by common Markov dynamics, we establish that the entire ensemble attains Poissonian steady states which are quantified by stationary Poissonian intensities and which hold valid also in the case of nonstationary Markov dynamics. The methodology is applied to a host of Markov dynamics, including Brownian motion, birth-death processes, random walks, geometric random walks, renewal processes, growth-collapse dynamics, decay-surge dynamics, Ito diffusions, and Langevin dynamics.

Poissonian steady states: From stationary densities to stationary intensities

NASA Astrophysics Data System (ADS)

Eliazar, Iddo

2012-10-01

Markov dynamics are the most elemental and omnipresent form of stochastic dynamics in the sciences, with applications ranging from physics to chemistry, from biology to evolution, and from economics to finance. Markov dynamics can be either stationary or nonstationary. Stationary Markov dynamics represent statistical steady states and are quantified by stationary densities. In this paper, we generalize the notion of steady state to the case of general Markov dynamics. Considering an ensemble of independent motions governed by common Markov dynamics, we establish that the entire ensemble attains Poissonian steady states which are quantified by stationary Poissonian intensities and which hold valid also in the case of nonstationary Markov dynamics. The methodology is applied to a host of Markov dynamics, including Brownian motion, birth-death processes, random walks, geometric random walks, renewal processes, growth-collapse dynamics, decay-surge dynamics, Ito diffusions, and Langevin dynamics.

On a Result for Finite Markov Chains

ERIC Educational Resources Information Center

Kulathinal, Sangita; Ghosh, Lagnojita

2006-01-01

In an undergraduate course on stochastic processes, Markov chains are discussed in great detail. Textbooks on stochastic processes provide interesting properties of finite Markov chains. This note discusses one such property regarding the number of steps in which a state is reachable or accessible from another state in a finite Markov chain with M…

The Markov blankets of life: autonomy, active inference and the free energy principle

PubMed Central

Palacios, Ensor; Friston, Karl; Kiverstein, Julian

2018-01-01

This work addresses the autonomous organization of biological systems. It does so by considering the boundaries of biological systems, from individual cells to Home sapiens, in terms of the presence of Markov blankets under the active inference scheme—a corollary of the free energy principle. A Markov blanket defines the boundaries of a system in a statistical sense. Here we consider how a collective of Markov blankets can self-assemble into a global system that itself has a Markov blanket; thereby providing an illustration of how autonomous systems can be understood as having layers of nested and self-sustaining boundaries. This allows us to show that: (i) any living system is a Markov blanketed system and (ii) the boundaries of such systems need not be co-extensive with the biophysical boundaries of a living organism. In other words, autonomous systems are hierarchically composed of Markov blankets of Markov blankets—all the way down to individual cells, all the way up to you and me, and all the way out to include elements of the local environment. PMID:29343629

Entanglement revival can occur only when the system-environment state is not a Markov state

NASA Astrophysics Data System (ADS)

Sargolzahi, Iman

2018-06-01

Markov states have been defined for tripartite quantum systems. In this paper, we generalize the definition of the Markov states to arbitrary multipartite case and find the general structure of an important subset of them, which we will call strong Markov states. In addition, we focus on an important property of the Markov states: If the initial state of the whole system-environment is a Markov state, then each localized dynamics of the whole system-environment reduces to a localized subdynamics of the system. This provides us a necessary condition for entanglement revival in an open quantum system: Entanglement revival can occur only when the system-environment state is not a Markov state. To illustrate (a part of) our results, we consider the case that the environment is modeled as classical. In this case, though the correlation between the system and the environment remains classical during the evolution, the change of the state of the system-environment, from its initial Markov state to a state which is not a Markov one, leads to the entanglement revival in the system. This shows that the non-Markovianity of a state is not equivalent to the existence of non-classical correlation in it, in general.

Wheat forecast economics effect study. [value of improved information on crop inventories, production, imports and exports

NASA Technical Reports Server (NTRS)

Mehra, R. K.; Rouhani, R.; Jones, S.; Schick, I.

1980-01-01

A model to assess the value of improved information regarding the inventories, productions, exports, and imports of crop on a worldwide basis is discussed. A previously proposed model is interpreted in a stochastic control setting and the underlying assumptions of the model are revealed. In solving the stochastic optimization problem, the Markov programming approach is much more powerful and exact as compared to the dynamic programming-simulation approach of the original model. The convergence of a dual variable Markov programming algorithm is shown to be fast and efficient. A computer program for the general model of multicountry-multiperiod is developed. As an example, the case of one country-two periods is treated and the results are presented in detail. A comparison with the original model results reveals certain interesting aspects of the algorithms and the dependence of the value of information on the incremental cost function.

[Prediction method of rural landscape pattern evolution based on life cycle: a case study of Jinjing Town, Hunan Province, China].

PubMed

Ji, Xiang; Liu, Li-Ming; Li, Hong-Qing

2014-11-01

Taking Jinjing Town in Dongting Lake area as a case, this paper analyzed the evolution of rural landscape patterns by means of life cycle theory, simulated the evolution cycle curve, and calculated its evolution period, then combining CA-Markov model, a complete prediction model was built based on the rule of rural landscape change. The results showed that rural settlement and paddy landscapes of Jinjing Town would change most in 2020, with the rural settlement landscape increased to 1194.01 hm2 and paddy landscape greatly reduced to 3090.24 hm2. The quantitative and spatial prediction accuracies of the model were up to 99.3% and 96.4%, respectively, being more explicit than single CA-Markov model. The prediction model of rural landscape patterns change proposed in this paper would be helpful for rural landscape planning in future.

Projection of postgraduate students flow with a smoothing matrix transition diagram of Markov chain

NASA Astrophysics Data System (ADS)

Rahim, Rahela; Ibrahim, Haslinda; Adnan, Farah Adibah

2013-04-01

This paper presents a case study of modeling postgraduate students flow at the College of Art and Sciences, Universiti Utara Malaysia. First, full time postgraduate students and the semester they were in are identified. Then administrative data were used to estimate the transitions between these semesters for the year 2001-2005 periods. Markov chain model is developed to calculate the -5 and -10 years projection of postgraduate students flow at the college. The optimization question addressed in this study is 'Which transitions would sustain the desired structure in the dynamic situation such as trend towards graduation?' The smoothed transition probabilities are proposed to estimate the transition probabilities matrix of 16 × 16. The results shows that using smoothed transition probabilities, the projection number of postgraduate students enrolled in the respective semesters are closer to actual than using the conventional steady states transition probabilities.

A nonstationary Markov transition model for computing the relative risk of dementia before death

PubMed Central

Yu, Lei; Griffith, William S.; Tyas, Suzanne L.; Snowdon, David A.; Kryscio, Richard J.

2010-01-01

This paper investigates the long-term behavior of the k-step transition probability matrix for a nonstationary discrete time Markov chain in the context of modeling transitions from intact cognition to dementia with mild cognitive impairment (MCI) and global impairment (GI) as intervening cognitive states. The authors derive formulas for the following absorption statistics: (1) the relative risk of absorption between competing absorbing states, and (2) the mean and variance of the number of visits among the transient states before absorption. Since absorption is not guaranteed, sufficient conditions are discussed to ensure that the substochastic matrix associated with transitions among transient states converges to zero in limit. Results are illustrated with an application to the Nun Study, a cohort of 678 participants, 75 to 107 years of age, followed longitudinally with up to ten cognitive assessments over a fifteen-year period. PMID:20087848

Detection of statistical asymmetries in non-stationary sign time series: Analysis of foreign exchange data

PubMed Central

Takayasu, Hideki; Takayasu, Misako

2017-01-01

We extend the concept of statistical symmetry as the invariance of a probability distribution under transformation to analyze binary sign time series data of price difference from the foreign exchange market. We model segments of the sign time series as Markov sequences and apply a local hypothesis test to evaluate the symmetries of independence and time reversion in different periods of the market. For the test, we derive the probability of a binary Markov process to generate a given set of number of symbol pairs. Using such analysis, we could not only segment the time series according the different behaviors but also characterize the segments in terms of statistical symmetries. As a particular result, we find that the foreign exchange market is essentially time reversible but this symmetry is broken when there is a strong external influence. PMID:28542208

An approximation formula for a class of Markov reliability models

NASA Technical Reports Server (NTRS)

White, A. L.

1984-01-01

A way of considering a small but often used class of reliability model and approximating algebraically the systems reliability is shown. The models considered are appropriate for redundant reconfigurable digital control systems that operate for a short period of time without maintenance, and for such systems the method gives a formula in terms of component fault rates, system recovery rates, and system operating time.

Chronic escitalopram treatment attenuated the accelerated rapid eye movement sleep transitions after selective rapid eye movement sleep deprivation: a model-based analysis using Markov chains.

PubMed

Kostyalik, Diána; Vas, Szilvia; Kátai, Zita; Kitka, Tamás; Gyertyán, István; Bagdy, Gyorgy; Tóthfalusi, László

2014-11-19

Shortened rapid eye movement (REM) sleep latency and increased REM sleep amount are presumed biological markers of depression. These sleep alterations are also observable in several animal models of depression as well as during the rebound sleep after selective REM sleep deprivation (RD). Furthermore, REM sleep fragmentation is typically associated with stress procedures and anxiety. The selective serotonin reuptake inhibitor (SSRI) antidepressants reduce REM sleep time and increase REM latency after acute dosing in normal condition and even during REM rebound following RD. However, their therapeutic outcome evolves only after weeks of treatment, and the effects of chronic treatment in REM-deprived animals have not been studied yet. Chronic escitalopram- (10 mg/kg/day, osmotic minipump for 24 days) or vehicle-treated rats were subjected to a 3-day-long RD on day 21 using the flower pot procedure or kept in home cage. On day 24, fronto-parietal electroencephalogram, electromyogram and motility were recorded in the first 2 h of the passive phase. The observed sleep patterns were characterized applying standard sleep metrics, by modelling the transitions between sleep phases using Markov chains and by spectral analysis. Based on Markov chain analysis, chronic escitalopram treatment attenuated the REM sleep fragmentation [accelerated transition rates between REM and non-REM (NREM) stages, decreased REM sleep residence time between two transitions] during the rebound sleep. Additionally, the antidepressant avoided the frequent awakenings during the first 30 min of recovery period. The spectral analysis showed that the SSRI prevented the RD-caused elevation in theta (5-9 Hz) power during slow-wave sleep. Conversely, based on the aggregate sleep metrics, escitalopram had only moderate effects and it did not significantly attenuate the REM rebound after RD. In conclusion, chronic SSRI treatment is capable of reducing several effects on sleep which might be the consequence of the sub-chronic stress caused by the flower pot method. These data might support the antidepressant activity of SSRIs, and may allude that investigating the rebound period following the flower pot protocol could be useful to detect antidepressant drug response. Markov analysis is a suitable method to study the sleep pattern.

Identifying and correcting non-Markov states in peptide conformational dynamics

NASA Astrophysics Data System (ADS)

Nerukh, Dmitry; Jensen, Christian H.; Glen, Robert C.

2010-02-01

Conformational transitions in proteins define their biological activity and can be investigated in detail using the Markov state model. The fundamental assumption on the transitions between the states, their Markov property, is critical in this framework. We test this assumption by analyzing the transitions obtained directly from the dynamics of a molecular dynamics simulated peptide valine-proline-alanine-leucine and states defined phenomenologically using clustering in dihedral space. We find that the transitions are Markovian at the time scale of ≈50 ps and longer. However, at the time scale of 30-40 ps the dynamics loses its Markov property. Our methodology reveals the mechanism that leads to non-Markov behavior. It also provides a way of regrouping the conformations into new states that now possess the required Markov property of their dynamics.

Representing Lumped Markov Chains by Minimal Polynomials over Field GF(q)

NASA Astrophysics Data System (ADS)

Zakharov, V. M.; Shalagin, S. V.; Eminov, B. F.

2018-05-01

A method has been proposed to represent lumped Markov chains by minimal polynomials over a finite field. The accuracy of representing lumped stochastic matrices, the law of lumped Markov chains depends linearly on the minimum degree of polynomials over field GF(q). The method allows constructing the realizations of lumped Markov chains on linear shift registers with a pre-defined “linear complexity”.

Behavior generation strategy of artificial behavioral system by self-learning paradigm for autonomous robot tasks

NASA Astrophysics Data System (ADS)

Dağlarli, Evren; Temeltaş, Hakan

2008-04-01

In this study, behavior generation and self-learning paradigms are investigated for the real-time applications of multi-goal mobile robot tasks. The method is capable to generate new behaviors and it combines them in order to achieve multi goal tasks. The proposed method is composed from three layers: Behavior Generating Module, Coordination Level and Emotion -Motivation Level. Last two levels use Hidden Markov models to manage dynamical structure of behaviors. The kinematics and dynamic model of the mobile robot with non-holonomic constraints are considered in the behavior based control architecture. The proposed method is tested on a four-wheel driven and four-wheel steered mobile robot with constraints in simulation environment and results are obtained successfully.

Bayesian selection of Markov models for symbol sequences: application to microsaccadic eye movements.

PubMed

Bettenbühl, Mario; Rusconi, Marco; Engbert, Ralf; Holschneider, Matthias

2012-01-01

Complex biological dynamics often generate sequences of discrete events which can be described as a Markov process. The order of the underlying Markovian stochastic process is fundamental for characterizing statistical dependencies within sequences. As an example for this class of biological systems, we investigate the Markov order of sequences of microsaccadic eye movements from human observers. We calculate the integrated likelihood of a given sequence for various orders of the Markov process and use this in a Bayesian framework for statistical inference on the Markov order. Our analysis shows that data from most participants are best explained by a first-order Markov process. This is compatible with recent findings of a statistical coupling of subsequent microsaccade orientations. Our method might prove to be useful for a broad class of biological systems.

Multi-site Stochastic Simulation of Daily Streamflow with Markov Chain and KNN Algorithm

NASA Astrophysics Data System (ADS)

Mathai, J.; Mujumdar, P.

2017-12-01

A key focus of this study is to develop a method which is physically consistent with the hydrologic processes that can capture short-term characteristics of daily hydrograph as well as the correlation of streamflow in temporal and spatial domains. In complex water resource systems, flow fluctuations at small time intervals require that discretisation be done at small time scales such as daily scales. Also, simultaneous generation of synthetic flows at different sites in the same basin are required. We propose a method to equip water managers with a streamflow generator within a stochastic streamflow simulation framework. The motivation for the proposed method is to generate sequences that extend beyond the variability represented in the historical record of streamflow time series. The method has two steps: In step 1, daily flow is generated independently at each station by a two-state Markov chain, with rising limb increments randomly sampled from a Gamma distribution and the falling limb modelled as exponential recession and in step 2, the streamflow generated in step 1 is input to a nonparametric K-nearest neighbor (KNN) time series bootstrap resampler. The KNN model, being data driven, does not require assumptions on the dependence structure of the time series. A major limitation of KNN based streamflow generators is that they do not produce new values, but merely reshuffle the historical data to generate realistic streamflow sequences. However, daily flow generated using the Markov chain approach is capable of generating a rich variety of streamflow sequences. Furthermore, the rising and falling limbs of daily hydrograph represent different physical processes, and hence they need to be modelled individually. Thus, our method combines the strengths of the two approaches. We show the utility of the method and improvement over the traditional KNN by simulating daily streamflow sequences at 7 locations in the Godavari River basin in India.

Markov Chain-Based Acute Effect Estimation of Air Pollution on Elder Asthma Hospitalization

PubMed Central

Luo, Li; Zhang, Fengyi; Sun, Lin; Li, Chunyang; Huang, Debin; Han, Gao; Wang, Bin

2017-01-01

Background Asthma caused substantial economic and health care burden and is susceptible to air pollution. Particularly, when it comes to elder asthma patient (older than 65), the phenomenon is more significant. The aim of this study is to investigate the Markov-based acute effects of air pollution on elder asthma hospitalizations, in forms of transition probabilities. Methods A retrospective, population-based study design was used to assess temporal patterns in hospitalizations for asthma in a region of Sichuan province, China. Approximately 12 million residents were covered during this period. Relative risk analysis and Markov chain model were employed on daily hospitalization state estimation. Results Among PM2.5, PM10, NO2, and SO2, only SO2 was significant. When air pollution is severe, the transition probability from a low-admission state (previous day) to high-admission state (next day) is 35.46%, while it is 20.08% when air pollution is mild. In particular, for female-cold subgroup, the counterparts are 30.06% and 0.01%, respectively. Conclusions SO2 was a significant risk factor for elder asthma hospitalization. When air pollution worsened, the transition probabilities from each state to high admission states increase dramatically. This phenomenon appeared more evidently, especially in female-cold subgroup (which is in cold season for female admissions). Based on our work, admission amount forecast, asthma intervention, and corresponding healthcare allocation can be done. PMID:29147496

Decoding and modelling of time series count data using Poisson hidden Markov model and Markov ordinal logistic regression models.

PubMed

Sebastian, Tunny; Jeyaseelan, Visalakshi; Jeyaseelan, Lakshmanan; Anandan, Shalini; George, Sebastian; Bangdiwala, Shrikant I

2018-01-01

Hidden Markov models are stochastic models in which the observations are assumed to follow a mixture distribution, but the parameters of the components are governed by a Markov chain which is unobservable. The issues related to the estimation of Poisson-hidden Markov models in which the observations are coming from mixture of Poisson distributions and the parameters of the component Poisson distributions are governed by an m-state Markov chain with an unknown transition probability matrix are explained here. These methods were applied to the data on Vibrio cholerae counts reported every month for 11-year span at Christian Medical College, Vellore, India. Using Viterbi algorithm, the best estimate of the state sequence was obtained and hence the transition probability matrix. The mean passage time between the states were estimated. The 95% confidence interval for the mean passage time was estimated via Monte Carlo simulation. The three hidden states of the estimated Markov chain are labelled as 'Low', 'Moderate' and 'High' with the mean counts of 1.4, 6.6 and 20.2 and the estimated average duration of stay of 3, 3 and 4 months, respectively. Environmental risk factors were studied using Markov ordinal logistic regression analysis. No significant association was found between disease severity levels and climate components.

Modeling of dialogue regimes of distance robot control

NASA Astrophysics Data System (ADS)

Larkin, E. V.; Privalov, A. N.

2017-02-01

Process of distance control of mobile robots is investigated. Petri-Markov net for modeling of dialogue regime is worked out. It is shown, that sequence of operations of next subjects: a human operator, a dialogue computer and an onboard computer may be simulated with use the theory of semi-Markov processes. From the semi-Markov process of the general form Markov process was obtained, which includes only states of transaction generation. It is shown, that a real transaction flow is the result of «concurrency» in states of Markov process. Iteration procedure for evaluation of transaction flow parameters, which takes into account effect of «concurrency», is proposed.

Algorithms for Discovery of Multiple Markov Boundaries

PubMed Central

Statnikov, Alexander; Lytkin, Nikita I.; Lemeire, Jan; Aliferis, Constantin F.

2013-01-01

Algorithms for Markov boundary discovery from data constitute an important recent development in machine learning, primarily because they offer a principled solution to the variable/feature selection problem and give insight on local causal structure. Over the last decade many sound algorithms have been proposed to identify a single Markov boundary of the response variable. Even though faithful distributions and, more broadly, distributions that satisfy the intersection property always have a single Markov boundary, other distributions/data sets may have multiple Markov boundaries of the response variable. The latter distributions/data sets are common in practical data-analytic applications, and there are several reasons why it is important to induce multiple Markov boundaries from such data. However, there are currently no sound and efficient algorithms that can accomplish this task. This paper describes a family of algorithms TIE* that can discover all Markov boundaries in a distribution. The broad applicability as well as efficiency of the new algorithmic family is demonstrated in an extensive benchmarking study that involved comparison with 26 state-of-the-art algorithms/variants in 15 data sets from a diversity of application domains. PMID:25285052

Statistical Analysis of Notational AFL Data Using Continuous Time Markov Chains

PubMed Central

Meyer, Denny; Forbes, Don; Clarke, Stephen R.

2006-01-01

Animal biologists commonly use continuous time Markov chain models to describe patterns of animal behaviour. In this paper we consider the use of these models for describing AFL football. In particular we test the assumptions for continuous time Markov chain models (CTMCs), with time, distance and speed values associated with each transition. Using a simple event categorisation it is found that a semi-Markov chain model is appropriate for this data. This validates the use of Markov Chains for future studies in which the outcomes of AFL matches are simulated. Key Points A comparison of four AFL matches suggests similarity in terms of transition probabilities for events and the mean times, distances and speeds associated with each transition. The Markov assumption appears to be valid. However, the speed, time and distance distributions associated with each transition are not exponential suggesting that semi-Markov model can be used to model and simulate play. Team identified events and directions associated with transitions are required to develop the model into a tool for the prediction of match outcomes. PMID:24357946

Statistical Analysis of Notational AFL Data Using Continuous Time Markov Chains.

PubMed

Meyer, Denny; Forbes, Don; Clarke, Stephen R

2006-01-01

Animal biologists commonly use continuous time Markov chain models to describe patterns of animal behaviour. In this paper we consider the use of these models for describing AFL football. In particular we test the assumptions for continuous time Markov chain models (CTMCs), with time, distance and speed values associated with each transition. Using a simple event categorisation it is found that a semi-Markov chain model is appropriate for this data. This validates the use of Markov Chains for future studies in which the outcomes of AFL matches are simulated. Key PointsA comparison of four AFL matches suggests similarity in terms of transition probabilities for events and the mean times, distances and speeds associated with each transition.The Markov assumption appears to be valid.However, the speed, time and distance distributions associated with each transition are not exponential suggesting that semi-Markov model can be used to model and simulate play.Team identified events and directions associated with transitions are required to develop the model into a tool for the prediction of match outcomes.

Data-driven confounder selection via Markov and Bayesian networks.

PubMed

Häggström, Jenny

2018-06-01

To unbiasedly estimate a causal effect on an outcome unconfoundedness is often assumed. If there is sufficient knowledge on the underlying causal structure then existing confounder selection criteria can be used to select subsets of the observed pretreatment covariates, X, sufficient for unconfoundedness, if such subsets exist. Here, estimation of these target subsets is considered when the underlying causal structure is unknown. The proposed method is to model the causal structure by a probabilistic graphical model, for example, a Markov or Bayesian network, estimate this graph from observed data and select the target subsets given the estimated graph. The approach is evaluated by simulation both in a high-dimensional setting where unconfoundedness holds given X and in a setting where unconfoundedness only holds given subsets of X. Several common target subsets are investigated and the selected subsets are compared with respect to accuracy in estimating the average causal effect. The proposed method is implemented with existing software that can easily handle high-dimensional data, in terms of large samples and large number of covariates. The results from the simulation study show that, if unconfoundedness holds given X, this approach is very successful in selecting the target subsets, outperforming alternative approaches based on random forests and LASSO, and that the subset estimating the target subset containing all causes of outcome yields smallest MSE in the average causal effect estimation. © 2017, The International Biometric Society.

A dynamic multi-scale Markov model based methodology for remaining life prediction

NASA Astrophysics Data System (ADS)

Yan, Jihong; Guo, Chaozhong; Wang, Xing

2011-05-01

The ability to accurately predict the remaining life of partially degraded components is crucial in prognostics. In this paper, a performance degradation index is designed using multi-feature fusion techniques to represent deterioration severities of facilities. Based on this indicator, an improved Markov model is proposed for remaining life prediction. Fuzzy C-Means (FCM) algorithm is employed to perform state division for Markov model in order to avoid the uncertainty of state division caused by the hard division approach. Considering the influence of both historical and real time data, a dynamic prediction method is introduced into Markov model by a weighted coefficient. Multi-scale theory is employed to solve the state division problem of multi-sample prediction. Consequently, a dynamic multi-scale Markov model is constructed. An experiment is designed based on a Bently-RK4 rotor testbed to validate the dynamic multi-scale Markov model, experimental results illustrate the effectiveness of the methodology.

Preliminary testing for the Markov property of the fifteen chromatin states of the Broad Histone Track.

PubMed

Lee, Kyung-Eun; Park, Hyun-Seok

2015-01-01

Epigenetic computational analyses based on Markov chains can integrate dependencies between regions in the genome that are directly adjacent. In this paper, the BED files of fifteen chromatin states of the Broad Histone Track of the ENCODE project are parsed, and comparative nucleotide frequencies of regional chromatin blocks are thoroughly analyzed to detect the Markov property in them. We perform various tests to examine the Markov property embedded in a frequency domain by checking for the presence of the Markov property in the various chromatin states. We apply these tests to each region of the fifteen chromatin states. The results of our simulation indicate that some of the chromatin states possess a stronger Markov property than others. We discuss the significance of our findings in statistical models of nucleotide sequences that are necessary for the computational analysis of functional units in noncoding DNA.

Cost-effectiveness analysis of allopurinol versus febuxostat in chronic gout patients: a U.S. payer perspective.

PubMed

Gandhi, Pranav K; Gentry, William M; Ma, Qinli; Bottorff, Michael B

2015-02-01

Gout is a chronic inflammatory condition associated with poor urate metabolism. Xanthine oxidase inhibitors such as allopurinol and febuxostat are recommended to reduce uric acid levels and to prevent gout attacks in adult patients. Under budget-driven constraints, health care payers are faced with the broader challenge of assessing the economic value of these agents for formulary placement. However, the economic value of allopurinol versus febuxostat has not be assessed in patients with gout over a 5-year time period in the United States. To evaluate the cost-effectiveness of allopurinol versus febuxostat in adult patients with gout over a 5-year time period from a U.S. health care payer's perspective. A Markov model was developed to compare the total direct costs and success of serum uric acid (sUA) level reduction associated with allopurinol and febuxostat. Treatment success was defined as patient achievement of a sUA level less than  6 mg/dL (0.36 mmol/L) at 6 months. Event probabilities were based on published phase III randomized clinical trials and included long-term sequelae from open-label extension studies. A hypothetical cohort of 1,000 adult gout patients with sUA levels of ≥ 8 mg/dL (0.48 mmol/L) who had received either allopurinol 300 mg or febuxostat 80 mg at model entry transitioned among the 4 health states defined by treatment success, treatment failure and switch, treatment dropout, and death. The length of each Markov cycle was 6 months. Costs were gathered from the RED BOOK, Medicare fee schedules, Healthcare Cost and Utilization Project's Nationwide Inpatient Sample, and for a limited number of inputs, expert consultation. Direct costs included treatment drug costs, costs for prophylaxis drugs, diagnostic laboratory tests, and the treatment and management of acute gout flare. Resource utilization was based on clinical evidence and expert consultation. All costs were inflated to 2014 U.S. dollars and were discounted at 3% in the base case. One-way sensitivity analysis and probabilistic sensitivity analyses (PSAs) were performed to assess the robustness of the results. The total per patient cost incurred over 5 years was $50,295 for febuxostat and $48,413 for allopurinol, with an incremental total cost of $1,882. The expected percentage of treatment success during the 5-year period was 72 for febuxostat and 42 for allopurinol, resulting in an incremental percentage of treatment success of 30. The estimated incremental cost-effectiveness ratio for febuxostat compared with allopurinol was $6,322 per treatment success over a 5-year time period. The one-way sensitivity analysis indicated that the results were sensitive to probability of treatment success for allopurinol, probability of treatment dropouts for both allopurinol and febuxostat, and the probability of failure and switch to allopurinol. PSAs demonstrated that at a willingness-to-pay threshold of $50,000 per treatment success, febuxostat was cost-effective compared with allopurinol. Febuxostat was found to be a cost-effective option compared with allopurinol based on a U.S. payer perspective.

Universal versus platelet reactivity assay-driven use of P2Y12 inhibitors in acute coronary syndrome patients: cost-effectiveness analyses for six European perspectives.

PubMed

Coleman, Craig I; Limone, Brendan L

2014-01-01

Platelet reactivity assays (PRAs) can predict patients' likely response to clopidogrel. As ticagrelor and prasugrel are typically considered first-line agents for acute coronary syndrome in Europe, we assessed the cost-effectiveness of universal compared to PRA-driven selection of these agents. A Markov model was used to calculate five-year costs (2013£/€), quality-adjusted life-years and incremental cost-effectiveness ratios (ICERs) for one-year of universal ticagrelor or prasugrel (given to all) compared to each agents' corresponding PRA-driven strategy (ticagrelor/prasugrel in those with high platelet reactivity [HPR, >208 on the VerifyNow P2Y12 assay], others given generic clopidogrel). We assumed patients had their index event at 65-70 years of age and had a 42.7% incidence of HPR 24-48 hours post-revascularisation. The analysis was conducted from the perspective of six countries (France, Germany, Italy, Spain, the Netherlands and United Kingdom) and used a one-year cycle length. Event data for P2Y12 inhibitors were taken from multinational randomised trials and adjusted using country-specific epidemiologic data. Neither universal ticagrelor nor prasugrel were found to be cost-effective (all ICERs >40,250€ or £36,600/QALY) compared to their corresponding PRA-driven strategies in any of the countries evaluated. Results were sensitive to differences in P2Y12 Inhibitors costs and drug-specific relative risks of major adverse cardiac events. Monte Carlo simulation suggested universal ticagrelor or prasugrel were cost-effective in only 25-44% and 11-17% of 10,000 iterations compared to their respective PRA-driven strategies, when applying a willingness-to-pay threshold = €30,000 or £20,000/QALY. In conclusion, the universal use of newer P2Y12 inhibitors is not likely cost-effective compared to PRA-driven strategies.

Hierarchical Markov blankets and adaptive active inference. Comment on "Answering Schrödinger's question: A free-energy formulation" by Maxwell James Désormeau Ramstead et al.

NASA Astrophysics Data System (ADS)

Kirchhoff, Michael

2018-03-01

Ramstead MJD, Badcock PB, Friston KJ. Answering Schrödinger's question: A free-energy formulation. Phys Life Rev 2018. https://doi.org/10.1016/j.plrev.2017.09.001 [this issue] motivate a multiscale characterisation of living systems in terms of hierarchically structured Markov blankets - a view of living systems as comprised of Markov blankets of Markov blankets [1-4]. It is effectively a treatment of what life is and how it is realised, cast in terms of how Markov blankets of living systems self-organise via active inference - a corollary of the free energy principle [5-7].

Modeling Hubble Space Telescope flight data by Q-Markov cover identification

NASA Technical Reports Server (NTRS)

Liu, K.; Skelton, R. E.; Sharkey, J. P.

1992-01-01

A state space model for the Hubble Space Telescope under the influence of unknown disturbances in orbit is presented. This model was obtained from flight data by applying the Q-Markov covariance equivalent realization identification algorithm. This state space model guarantees the match of the first Q-Markov parameters and covariance parameters of the Hubble system. The flight data were partitioned into high- and low-frequency components for more efficient Q-Markov cover modeling, to reduce some computational difficulties of the Q-Markov cover algorithm. This identification revealed more than 20 lightly damped modes within the bandwidth of the attitude control system. Comparisons with the analytical (TREETOPS) model are also included.

Scaling Limit of Symmetric Random Walk in High-Contrast Periodic Environment

NASA Astrophysics Data System (ADS)

Piatnitski, A.; Zhizhina, E.

2017-11-01

The paper deals with the asymptotic properties of a symmetric random walk in a high contrast periodic medium in Z^d, d≥1. From the existing homogenization results it follows that under diffusive scaling the limit behaviour of this random walk need not be Markovian. The goal of this work is to show that if in addition to the coordinate of the random walk in Z^d we introduce an extra variable that characterizes the position of the random walk inside the period then the limit dynamics of this two-component process is Markov. We describe the limit process and observe that the components of the limit process are coupled. We also prove the convergence in the path space for the said random walk.

Object-based change detection method using refined Markov random field

NASA Astrophysics Data System (ADS)

Peng, Daifeng; Zhang, Yongjun

2017-01-01

In order to fully consider the local spatial constraints between neighboring objects in object-based change detection (OBCD), an OBCD approach is presented by introducing a refined Markov random field (MRF). First, two periods of images are stacked and segmented to produce image objects. Second, object spectral and textual histogram features are extracted and G-statistic is implemented to measure the distance among different histogram distributions. Meanwhile, object heterogeneity is calculated by combining spectral and textual histogram distance using adaptive weight. Third, an expectation-maximization algorithm is applied for determining the change category of each object and the initial change map is then generated. Finally, a refined change map is produced by employing the proposed refined object-based MRF method. Three experiments were conducted and compared with some state-of-the-art unsupervised OBCD methods to evaluate the effectiveness of the proposed method. Experimental results demonstrate that the proposed method obtains the highest accuracy among the methods used in this paper, which confirms its validness and effectiveness in OBCD.

Upscaling transport of a reacting solute through a peridocially converging-diverging channel at pre-asymptotic times

NASA Astrophysics Data System (ADS)

Sund, Nicole L.; Bolster, Diogo; Dawson, Clint

2015-11-01

In this study we extend the Spatial Markov model, which has been successfully used to upscale conservative transport across a diverse range of porous media flows, to test if it can accurately upscale reactive transport, defined by a spatially heterogeneous first order degradation rate. We test the model in a well known highly simplified geometry, commonly considered as an idealized pore or fracture structure, a periodic channel with wavy boundaries. The edges of the flow domain have a layer through which there is no flow, but in which diffusion of a solute still occurs. Reactions are confined to this region. We demonstrate that the Spatial Markov model, an upscaled random walk model that enforces correlation between successive jumps, can reproduce breakthrough curves measured from microscale simulations that explicitly resolve all pertinent processes. We also demonstrate that a similar random walk model that does not enforce successive correlations is unable to reproduce all features of the measured breakthrough curves.

Spectral analysis and markov switching model of Indonesia business cycle

NASA Astrophysics Data System (ADS)

Fajar, Muhammad; Darwis, Sutawanir; Darmawan, Gumgum

2017-03-01

This study aims to investigate the Indonesia business cycle encompassing the determination of smoothing parameter (λ) on Hodrick-Prescott filter. Subsequently, the components of the filter output cycles were analyzed using a spectral method useful to know its characteristics, and Markov switching regime modeling is made to forecast the probability recession and expansion regimes. The data used in the study is real GDP (1983Q1 - 2016Q2). The results of the study are: a) Hodrick-Prescott filter on real GDP of Indonesia to be optimal when the value of the smoothing parameter is 988.474, b) Indonesia business cycle has amplitude varies between±0.0071 to±0.01024, and the duration is between 4 to 22 quarters, c) the business cycle can be modelled by MSIV-AR (2) but regime periodization is generated this model not perfect exactly with real regime periodzation, and d) Based on the model MSIV-AR (2) obtained long-term probabilities in the expansion regime: 0.4858 and in the recession regime: 0.5142.

LECTURES ON GAME THEORY, MARKOV CHAINS, AND RELATED TOPICS

DOE Office of Scientific and Technical Information (OSTI.GOV)

Thompson, G L

1958-03-01

Notes on nine lectures delivered at Sandin Corporation in August 1957 are given. Part one contains the manuscript of a paper concerning a judging problem. Part two is concerned with finite Markov-chain theory amd discusses regular Markov chains, absorbing Markov chains, the classification of states, application to the Leontief input-output model, and semimartingales. Part three contains notes on game theory and covers matrix games, the effect of psychological attitudes on the outcomes of games, extensive games, amd matrix theory applied to mathematical economics. (auth)

Markov chains: computing limit existence and approximations with DNA.

PubMed

Cardona, M; Colomer, M A; Conde, J; Miret, J M; Miró, J; Zaragoza, A

2005-09-01

We present two algorithms to perform computations over Markov chains. The first one determines whether the sequence of powers of the transition matrix of a Markov chain converges or not to a limit matrix. If it does converge, the second algorithm enables us to estimate this limit. The combination of these algorithms allows the computation of a limit using DNA computing. In this sense, we have encoded the states and the transition probabilities using strands of DNA for generating paths of the Markov chain.

Markov models in dentistry: application to resin-bonded bridges and review of the literature.

PubMed

Mahl, Dominik; Marinello, Carlo P; Sendi, Pedram

2012-10-01

Markov models are mathematical models that can be used to describe disease progression and evaluate the cost-effectiveness of medical interventions. Markov models allow projecting clinical and economic outcomes into the future and are therefore frequently used to estimate long-term outcomes of medical interventions. The purpose of this paper is to demonstrate its use in dentistry, using the example of resin-bonded bridges to replace missing teeth, and to review the literature. We used literature data and a four-state Markov model to project long-term outcomes of resin-bonded bridges over a time horizon of 60 years. In addition, the literature was searched in PubMed Medline for research articles on the application of Markov models in dentistry.

The generalization ability of SVM classification based on Markov sampling.

PubMed

Xu, Jie; Tang, Yuan Yan; Zou, Bin; Xu, Zongben; Li, Luoqing; Lu, Yang; Zhang, Baochang

2015-06-01

The previously known works studying the generalization ability of support vector machine classification (SVMC) algorithm are usually based on the assumption of independent and identically distributed samples. In this paper, we go far beyond this classical framework by studying the generalization ability of SVMC based on uniformly ergodic Markov chain (u.e.M.c.) samples. We analyze the excess misclassification error of SVMC based on u.e.M.c. samples, and obtain the optimal learning rate of SVMC for u.e.M.c. We also introduce a new Markov sampling algorithm for SVMC to generate u.e.M.c. samples from given dataset, and present the numerical studies on the learning performance of SVMC based on Markov sampling for benchmark datasets. The numerical studies show that the SVMC based on Markov sampling not only has better generalization ability as the number of training samples are bigger, but also the classifiers based on Markov sampling are sparsity when the size of dataset is bigger with regard to the input dimension.

Grey-Markov prediction model based on background value optimization and central-point triangular whitenization weight function

NASA Astrophysics Data System (ADS)

Ye, Jing; Dang, Yaoguo; Li, Bingjun

2018-01-01

Grey-Markov forecasting model is a combination of grey prediction model and Markov chain which show obvious optimization effects for data sequences with characteristics of non-stationary and volatility. However, the state division process in traditional Grey-Markov forecasting model is mostly based on subjective real numbers that immediately affects the accuracy of forecasting values. To seek the solution, this paper introduces the central-point triangular whitenization weight function in state division to calculate possibilities of research values in each state which reflect preference degrees in different states in an objective way. On the other hand, background value optimization is applied in the traditional grey model to generate better fitting data. By this means, the improved Grey-Markov forecasting model is built. Finally, taking the grain production in Henan Province as an example, it verifies this model's validity by comparing with GM(1,1) based on background value optimization and the traditional Grey-Markov forecasting model.

Caliber Corrected Markov Modeling (C2M2): Correcting Equilibrium Markov Models.

PubMed

Dixit, Purushottam D; Dill, Ken A

2018-02-13

Rate processes are often modeled using Markov State Models (MSMs). Suppose you know a prior MSM and then learn that your prediction of some particular observable rate is wrong. What is the best way to correct the whole MSM? For example, molecular dynamics simulations of protein folding may sample many microstates, possibly giving correct pathways through them while also giving the wrong overall folding rate when compared to experiment. Here, we describe Caliber Corrected Markov Modeling (C 2 M 2 ), an approach based on the principle of maximum entropy for updating a Markov model by imposing state- and trajectory-based constraints. We show that such corrections are equivalent to asserting position-dependent diffusion coefficients in continuous-time continuous-space Markov processes modeled by a Smoluchowski equation. We derive the functional form of the diffusion coefficient explicitly in terms of the trajectory-based constraints. We illustrate with examples of 2D particle diffusion and an overdamped harmonic oscillator.

Phasic Triplet Markov Chains.

PubMed

El Yazid Boudaren, Mohamed; Monfrini, Emmanuel; Pieczynski, Wojciech; Aïssani, Amar

2014-11-01

Hidden Markov chains have been shown to be inadequate for data modeling under some complex conditions. In this work, we address the problem of statistical modeling of phenomena involving two heterogeneous system states. Such phenomena may arise in biology or communications, among other fields. Namely, we consider that a sequence of meaningful words is to be searched within a whole observation that also contains arbitrary one-by-one symbols. Moreover, a word may be interrupted at some site to be carried on later. Applying plain hidden Markov chains to such data, while ignoring their specificity, yields unsatisfactory results. The Phasic triplet Markov chain, proposed in this paper, overcomes this difficulty by means of an auxiliary underlying process in accordance with the triplet Markov chains theory. Related Bayesian restoration techniques and parameters estimation procedures according to the new model are then described. Finally, to assess the performance of the proposed model against the conventional hidden Markov chain model, experiments are conducted on synthetic and real data.

Markov-modulated Markov chains and the covarion process of molecular evolution.

PubMed

Galtier, N; Jean-Marie, A

2004-01-01

The covarion (or site specific rate variation, SSRV) process of biological sequence evolution is a process by which the evolutionary rate of a nucleotide/amino acid/codon position can change in time. In this paper, we introduce time-continuous, space-discrete, Markov-modulated Markov chains as a model for representing SSRV processes, generalizing existing theory to any model of rate change. We propose a fast algorithm for diagonalizing the generator matrix of relevant Markov-modulated Markov processes. This algorithm makes phylogeny likelihood calculation tractable even for a large number of rate classes and a large number of states, so that SSRV models become applicable to amino acid or codon sequence datasets. Using this algorithm, we investigate the accuracy of the discrete approximation to the Gamma distribution of evolutionary rates, widely used in molecular phylogeny. We show that a relatively large number of classes is required to achieve accurate approximation of the exact likelihood when the number of analyzed sequences exceeds 20, both under the SSRV and among site rate variation (ASRV) models.

Nonadiabatic effects in periodically driven dissipative open quantum systems

NASA Astrophysics Data System (ADS)

Reimer, Viktor; Pedersen, Kim G. L.; Tanger, Niklas; Pletyukhov, Mikhail; Gritsev, Vladimir

2018-04-01

We present a general method to calculate the periodic steady state of a driven-dissipative system coupled to a transmission line (and more generally, to a reservoir) under periodic modulation of its parameters. Using Floquet's theorem, we formulate the differential equation for the system's density operator which has to be solved for a single period of modulation. On this basis we also provide systematic expansions in both the adiabatic and high-frequency regime. Applying our method to three different systems—two- and three-level models as well as the driven nonlinear cavity—we propose periodic modulation protocols of parameters leading to a temporary suppression of effective dissipation rates, and study the arising nonadiabatic features in the response of these systems.

Resonances in a periodically driven bosonic system.

PubMed

Quelle, Anton; Smith, Cristiane Morais

2017-11-01

Periodically driven systems are a common topic in modern physics. In optical lattices specifically, driving is at the origin of many interesting phenomena. However, energy is not conserved in driven systems, and under periodic driving, heating of a system is a real concern. In an effort to better understand this phenomenon, the heating of single-band systems has been studied, with a focus on disorder- and interaction-induced effects, such as many-body localization. Nevertheless, driven systems occur in a much wider context than this, leaving room for further research. Here, we fill this gap by studying a noninteracting model, characterized by discrete, periodically spaced energy levels that are unbounded from above. We couple these energy levels resonantly through a periodic drive, and discuss the heating dynamics of this system as a function of the driving protocol. In this way, we show that a combination of stimulated emission and absorption causes the presence of resonant stable states. This will serve to elucidate the conditions under which resonant driving causes heating in quantum systems.

Resonances in a periodically driven bosonic system

NASA Astrophysics Data System (ADS)

Quelle, Anton; Smith, Cristiane Morais

2017-11-01

Periodically driven systems are a common topic in modern physics. In optical lattices specifically, driving is at the origin of many interesting phenomena. However, energy is not conserved in driven systems, and under periodic driving, heating of a system is a real concern. In an effort to better understand this phenomenon, the heating of single-band systems has been studied, with a focus on disorder- and interaction-induced effects, such as many-body localization. Nevertheless, driven systems occur in a much wider context than this, leaving room for further research. Here, we fill this gap by studying a noninteracting model, characterized by discrete, periodically spaced energy levels that are unbounded from above. We couple these energy levels resonantly through a periodic drive, and discuss the heating dynamics of this system as a function of the driving protocol. In this way, we show that a combination of stimulated emission and absorption causes the presence of resonant stable states. This will serve to elucidate the conditions under which resonant driving causes heating in quantum systems.

Cost effectiveness of adding clostridial collagenase ointment to selective debridement in individuals with stage IV pressure ulcers.

PubMed

Carter, Marissa J; Gilligan, Adrienne M; Waycaster, Curtis R; Schaum, Kathleen; Fife, Caroline E

2017-03-01

The purpose of this study was to determine the cost effectiveness (from a payer's perspective) of adding clostridial collagenase ointment (CCO) to selective debridement compared with selective debridement alone (non-CCO) in the treatment of stage IV pressure ulcers among patients identified from the US Wound Registry. A 3-state Markov model was developed to determine costs and outcomes between the CCO and non-CCO groups over a 2-year time horizon. Outcome data were derived from a retrospective clinical study and included the proportion of pressure ulcers that were closed (epithelialized) over 2 years and the time to wound closure. Transition probabilities for the Markov states were estimated from the clinical study. In the Markov model, the clinical outcome is presented as ulcer-free weeks, which represents the time the wound is in the epithelialized state. Costs for each 4-week cycle were based on frequencies of clinic visits, debridement, and CCO application rates from the clinical study. The final model outputs were cumulative costs (in US dollars), clinical outcome (ulcer-free weeks), and incremental cost-effectiveness ratio (ICER) at 2 years. Compared with the non-CCO group, the CCO group incurred lower costs ($11,151 vs $17,596) and greater benefits (33.9 vs 16.8 ulcer-free weeks), resulting in an economically dominant ICER of -$375 per ulcer. Thus, for each additional ulcer-free week that can be gained, there is a concurrent cost savings of $375 if CCO treatment is selected. Over a 2-year period, an additional 17.2 ulcer-free weeks can be gained with concurrent cost savings of $6,445 for each patient. In this Markov model based on real-world data from the US Wound Registry, the addition of CCO to selective debridement in the treatment of pressure ulcers was economically dominant over selective debridement alone, resulting in greater benefit to the patient at lower cost.

Species interactions and the effects of climate variability on a wetland amphibian metacommunity

USGS Publications Warehouse

Davis, Courtney L.; Miller, David A.W.; Walls, Susan C.; Barichivich, William J.; Riley, Jeffrey W.; Brown, Mary E.

2017-01-01

Disentangling the role that multiple interacting factors have on species responses to shifting climate poses a significant challenge. However, our ability to do so is of utmost importance to predict the effects of climate change on species distributions. We examined how populations of three species of wetland-breeding amphibians, which varied in life history requirements, responded to a six-year period of extremely variable precipitation. This interval was punctuated by both extensive drought and heavy precipitation and flooding, providing a natural experiment to measure community responses to environmental perturbations. We estimated occurrence dynamics using a discrete hidden Markov modeling approach that incorporated information regarding habitat state and predator–prey interactions. This approach allowed us to measure how metapopulation dynamics of each amphibian species was affected by interactions among weather, wetland hydroperiod, and co-occurrence with fish predators. The pig frog, a generalist, proved most resistant to perturbations, with both colonization and persistence being unaffected by seasonal variation in precipitation or co-occurrence with fishes. The ornate chorus frog, an ephemeral wetland specialist, responded positively to periods of drought owing to increased persistence and colonization rates during periods of low-rainfall. Low probabilities of occurrence of the ornate chorus frog in long-duration wetlands were driven by interactions with predators due to low colonization rates when fishes were present. The mole salamander was most sensitive to shifts in water availability. In our study area, this species never occurred in short-duration wetlands and persistence probabilities decreased during periods of drought. At the same time, negative effects occurred with extreme precipitation because flooding facilitated colonization of fishes to isolated wetlands and mole salamanders did not colonize wetlands once fishes were present. We demonstrate that the effects of changes in water availability depend on interactions with predators and wetland type and are influenced by the life history of each of our species. The dynamic species occurrence modeling approach we used offers promise for other systems when the goal is to disentangle the complex interactions that determine species responses to environmental variability.

Analysing grouping of nucleotides in DNA sequences using lumped processes constructed from Markov chains.

PubMed

Guédon, Yann; d'Aubenton-Carafa, Yves; Thermes, Claude

2006-03-01

The most commonly used models for analysing local dependencies in DNA sequences are (high-order) Markov chains. Incorporating knowledge relative to the possible grouping of the nucleotides enables to define dedicated sub-classes of Markov chains. The problem of formulating lumpability hypotheses for a Markov chain is therefore addressed. In the classical approach to lumpability, this problem can be formulated as the determination of an appropriate state space (smaller than the original state space) such that the lumped chain defined on this state space retains the Markov property. We propose a different perspective on lumpability where the state space is fixed and the partitioning of this state space is represented by a one-to-many probabilistic function within a two-level stochastic process. Three nested classes of lumped processes can be defined in this way as sub-classes of first-order Markov chains. These lumped processes enable parsimonious reparameterizations of Markov chains that help to reveal relevant partitions of the state space. Characterizations of the lumped processes on the original transition probability matrix are derived. Different model selection methods relying either on hypothesis testing or on penalized log-likelihood criteria are presented as well as extensions to lumped processes constructed from high-order Markov chains. The relevance of the proposed approach to lumpability is illustrated by the analysis of DNA sequences. In particular, the use of lumped processes enables to highlight differences between intronic sequences and gene untranslated region sequences.

Markov processes for the prediction of aircraft noise effects on sleep.

PubMed

Basner, Mathias; Siebert, Uwe

2010-01-01

Aircraft noise disturbs sleep and impairs recuperation. Authorities plan to expand Frankfurt airport. To quantitatively assess the effects of a traffic curfew (11 PM to 5 AM) at Frankfurt Airport on sleep structure. Experimental sleep study; polysomnography for 13 consecutive nights. Sleep laboratory. Subjects. 128 healthy subjects, mean age (SD) 38 (13) years, range 19 to 65, 59% female. Intervention. Exposure to aircraft noise via loudspeakers. A 6-state Markov state transition sleep model was used to simulate 3 noise scenarios with first-order Monte Carlo simulations: 1) 2005 traffic at Frankfurt Airport, 2) as simulation 1 but flights between 11 PM and 5 AM cancelled, and 3) as simulation 2, with flights between 11 PM and 5 AM from simulation 1 rescheduled to periods before 11 PM and after 5 AM. Probabilities for transitions between sleep stages were estimated with autoregressive multinomial logistic regression. Compared to a night without curfew, models indicate small improvements in sleep structure in nights with curfew, even if all traffic is rescheduled to periods before and after the curfew period. For those who go to bed before 10:30 PM or after 1 AM, this benefit is likely to be offset by the expected increase of air traffic during late evening and early morning hours. Limitations. Limited ecologic validity due to laboratory setting and subject sample. According to the decision analysis, it is unlikely that the proposed curfew at Frankfurt Airport substantially benefits sleep structure. Extensions of the model could be used to evaluate or propose alternative air traffic regulation strategies for Frankfurt Airport.

Simulation of land use change and effect on potential deforestation using Markov Chain - Cellular Automata

NASA Astrophysics Data System (ADS)

Mujiono, Indra, T. L.; Harmantyo, D.; Rukmana, I. P.; Nadia, Z.

2017-07-01

The purpose of this study was to simulate land use change in 1996-2016 and its prediction in 2035 as well as its potential to deforestation. Both of these purposes were obtained through modeling analysis using Markov Chain Cellular Automata. This modeling method was considered important for understanding the causes and impacts. Based on the analysis, the land use change between 1996 to 2007 has caused forest loss (the region and non-region) covering an area of 62,012 ha. While in the period of 2007 to 2016, the change has lead to the east side of the slope grade of 0-15 percent and an altitude between 500-1000 meters above sea level. In this period, plantation area has increased by 50,822 ha, while the forest area has reduced from 80,038 ha. In a period of 20 years, North Bengkulu Regency has lost the forest area of 80,038 ha. The amount of intervention against forest suggested the potential for deforestation in this area. Simulation of land use change in 2035 did not indicate significant deforestation due to the limited land on physical factors such as slope and elevation. However, it should be noted that, in 2035, the area of conservation forest was reduced by 16,793 ha (29 %), while the areas of protected and production forest were reduced by 4,933 ha (19 %) and 2,114 ha (3 %), respectively. Land use change is a serious threat of deforestation, especially in forest areas in North Bengkulu Regency, where any decline in forest area means the addition of plantation area.

Vulnerability to paroxysmal oscillations in delayed neural networks: A basis for nocturnal frontal lobe epilepsy?

NASA Astrophysics Data System (ADS)

Quan, Austin; Osorio, Ivan; Ohira, Toru; Milton, John

2011-12-01

Resonance can occur in bistable dynamical systems due to the interplay between noise and delay (τ) in the absence of a periodic input. We investigate resonance in a two-neuron model with mutual time-delayed inhibitory feedback. For appropriate choices of the parameters and inputs three fixed-point attractors co-exist: two are stable and one is unstable. In the absence of noise, delay-induced transient oscillations (referred to herein as DITOs) arise whenever the initial function is tuned sufficiently close to the unstable fixed-point. In the presence of noisy perturbations, DITOs arise spontaneously. Since the correlation time for the stationary dynamics is ˜τ, we approximated a higher order Markov process by a three-state Markov chain model by rescaling time as t → 2sτ, identifying the states based on whether the sub-intervals were completely confined to one basin of attraction (the two stable attractors) or straddled the separatrix, and then determining the transition probability matrix empirically. The resultant Markov chain model captured the switching behaviors including the statistical properties of the DITOs. Our observations indicate that time-delayed and noisy bistable dynamical systems are prone to generate DITOs as switches between the two attractors occur. Bistable systems arise transiently in situations when one attractor is gradually replaced by another. This may explain, for example, why seizures in certain epileptic syndromes tend to occur as sleep stages change.

Building Simple Hidden Markov Models. Classroom Notes

ERIC Educational Resources Information Center

Ching, Wai-Ki; Ng, Michael K.

2004-01-01

Hidden Markov models (HMMs) are widely used in bioinformatics, speech recognition and many other areas. This note presents HMMs via the framework of classical Markov chain models. A simple example is given to illustrate the model. An estimation method for the transition probabilities of the hidden states is also discussed.

Using Games to Teach Markov Chains

ERIC Educational Resources Information Center

Johnson, Roger W.

2003-01-01

Games are promoted as examples for classroom discussion of stationary Markov chains. In a game context Markov chain terminology and results are made concrete, interesting, and entertaining. Game length for several-player games such as "Hi Ho! Cherry-O" and "Chutes and Ladders" is investigated and new, simple formulas are given. Slight…

Sampling rare fluctuations of discrete-time Markov chains

NASA Astrophysics Data System (ADS)

Whitelam, Stephen

2018-03-01

We describe a simple method that can be used to sample the rare fluctuations of discrete-time Markov chains. We focus on the case of Markov chains with well-defined steady-state measures, and derive expressions for the large-deviation rate functions (and upper bounds on such functions) for dynamical quantities extensive in the length of the Markov chain. We illustrate the method using a series of simple examples, and use it to study the fluctuations of a lattice-based model of active matter that can undergo motility-induced phase separation.

Sampling rare fluctuations of discrete-time Markov chains.

PubMed

Whitelam, Stephen

2018-03-01

We describe a simple method that can be used to sample the rare fluctuations of discrete-time Markov chains. We focus on the case of Markov chains with well-defined steady-state measures, and derive expressions for the large-deviation rate functions (and upper bounds on such functions) for dynamical quantities extensive in the length of the Markov chain. We illustrate the method using a series of simple examples, and use it to study the fluctuations of a lattice-based model of active matter that can undergo motility-induced phase separation.

Enhancement of Markov chain model by integrating exponential smoothing: A case study on Muslims marriage and divorce

NASA Astrophysics Data System (ADS)

Jamaluddin, Fadhilah; Rahim, Rahela Abdul

2015-12-01

Markov Chain has been introduced since the 1913 for the purpose of studying the flow of data for a consecutive number of years of the data and also forecasting. The important feature in Markov Chain is obtaining the accurate Transition Probability Matrix (TPM). However to obtain the suitable TPM is hard especially in involving long-term modeling due to unavailability of data. This paper aims to enhance the classical Markov Chain by introducing Exponential Smoothing technique in developing the appropriate TPM.

Decentralized learning in Markov games.

PubMed

Vrancx, Peter; Verbeeck, Katja; Nowé, Ann

2008-08-01

Learning automata (LA) were recently shown to be valuable tools for designing multiagent reinforcement learning algorithms. One of the principal contributions of the LA theory is that a set of decentralized independent LA is able to control a finite Markov chain with unknown transition probabilities and rewards. In this paper, we propose to extend this algorithm to Markov games--a straightforward extension of single-agent Markov decision problems to distributed multiagent decision problems. We show that under the same ergodic assumptions of the original theorem, the extended algorithm will converge to a pure equilibrium point between agent policies.

The generalization ability of online SVM classification based on Markov sampling.

PubMed

Xu, Jie; Yan Tang, Yuan; Zou, Bin; Xu, Zongben; Li, Luoqing; Lu, Yang

2015-03-01

In this paper, we consider online support vector machine (SVM) classification learning algorithms with uniformly ergodic Markov chain (u.e.M.c.) samples. We establish the bound on the misclassification error of an online SVM classification algorithm with u.e.M.c. samples based on reproducing kernel Hilbert spaces and obtain a satisfactory convergence rate. We also introduce a novel online SVM classification algorithm based on Markov sampling, and present the numerical studies on the learning ability of online SVM classification based on Markov sampling for benchmark repository. The numerical studies show that the learning performance of the online SVM classification algorithm based on Markov sampling is better than that of classical online SVM classification based on random sampling as the size of training samples is larger.

NonMarkov Ito Processes with 1- state memory

NASA Astrophysics Data System (ADS)

McCauley, Joseph L.

2010-08-01

A Markov process, by definition, cannot depend on any previous state other than the last observed state. An Ito process implies the Fokker-Planck and Kolmogorov backward time partial differential eqns. for transition densities, which in turn imply the Chapman-Kolmogorov eqn., but without requiring the Markov condition. We present a class of Ito process superficially resembling Markov processes, but with 1-state memory. In finance, such processes would obey the efficient market hypothesis up through the level of pair correlations. These stochastic processes have been mislabeled in recent literature as 'nonlinear Markov processes'. Inspired by Doob and Feller, who pointed out that the ChapmanKolmogorov eqn. is not restricted to Markov processes, we exhibit a Gaussian Ito transition density with 1-state memory in the drift coefficient that satisfies both of Kolmogorov's partial differential eqns. and also the Chapman-Kolmogorov eqn. In addition, we show that three of the examples from McKean's seminal 1966 paper are also nonMarkov Ito processes. Last, we show that the transition density of the generalized Black-Scholes type partial differential eqn. describes a martingale, and satisfies the ChapmanKolmogorov eqn. This leads to the shortest-known proof that the Green function of the Black-Scholes eqn. with variable diffusion coefficient provides the so-called martingale measure of option pricing.

Stochastic dynamics of extended objects in driven systems II: Current quantization in the low-temperature limit

NASA Astrophysics Data System (ADS)

Catanzaro, Michael J.; Chernyak, Vladimir Y.; Klein, John R.

2016-12-01

Driven Langevin processes have appeared in a variety of fields due to the relevance of natural phenomena having both deterministic and stochastic effects. The stochastic currents and fluxes in these systems provide a convenient set of observables to describe their non-equilibrium steady states. Here we consider stochastic motion of a (k - 1) -dimensional object, which sweeps out a k-dimensional trajectory, and gives rise to a higher k-dimensional current. By employing the low-temperature (low-noise) limit, we reduce the problem to a discrete Markov chain model on a CW complex, a topological construction which generalizes the notion of a graph. This reduction allows the mean fluxes and currents of the process to be expressed in terms of solutions to the discrete Supersymmetric Fokker-Planck (SFP) equation. Taking the adiabatic limit, we show that generic driving leads to rational quantization of the generated higher dimensional current. The latter is achieved by implementing the recently developed tools, coined the higher-dimensional Kirchhoff tree and co-tree theorems. This extends the study of motion of extended objects in the continuous setting performed in the prequel (Catanzaro et al.) to this manuscript.

Period doubling in period-one steady states

NASA Astrophysics Data System (ADS)

Wang, Reuben R. W.; Xing, Bo; Carlo, Gabriel G.; Poletti, Dario

2018-02-01

Nonlinear classical dissipative systems present a rich phenomenology in their "route to chaos," including period doubling, i.e., the system evolves with a period which is twice that of the driving. However, typically the attractor of a periodically driven quantum open system evolves with a period which exactly matches that of the driving. Here, we analyze a periodically driven many-body open quantum system whose classical correspondent presents period doubling. We show that by studying the dynamical correlations, it is possible to show the occurrence of period doubling in the quantum (period-one) steady state. We also discuss that such systems are natural candidates for clean and intrinsically robust Floquet time crystals.

Master equation for She-Leveque scaling and its classification in terms of other Markov models of developed turbulence

NASA Astrophysics Data System (ADS)

Nickelsen, Daniel

2017-07-01

The statistics of velocity increments in homogeneous and isotropic turbulence exhibit universal features in the limit of infinite Reynolds numbers. After Kolmogorov’s scaling law from 1941, many turbulence models aim for capturing these universal features, some are known to have an equivalent formulation in terms of Markov processes. We derive the Markov process equivalent to the particularly successful scaling law postulated by She and Leveque. The Markov process is a jump process for velocity increments u(r) in scale r in which the jumps occur randomly but with deterministic width in u. From its master equation we establish a prescription to simulate the She-Leveque process and compare it with Kolmogorov scaling. To put the She-Leveque process into the context of other established turbulence models on the Markov level, we derive a diffusion process for u(r) using two properties of the Navier-Stokes equation. This diffusion process already includes Kolmogorov scaling, extended self-similarity and a class of random cascade models. The fluctuation theorem of this Markov process implies a ‘second law’ that puts a loose bound on the multipliers of the random cascade models. This bound explicitly allows for instances of inverse cascades, which are necessary to satisfy the fluctuation theorem. By adding a jump process to the diffusion process, we go beyond Kolmogorov scaling and formulate the most general scaling law for the class of Markov processes having both diffusion and jump parts. This Markov scaling law includes She-Leveque scaling and a scaling law derived by Yakhot.

Efficient analysis of stochastic gene dynamics in the non-adiabatic regime using piecewise deterministic Markov processes

PubMed Central

2018-01-01

Single-cell experiments show that gene expression is stochastic and bursty, a feature that can emerge from slow switching between promoter states with different activities. In addition to slow chromatin and/or DNA looping dynamics, one source of long-lived promoter states is the slow binding and unbinding kinetics of transcription factors to promoters, i.e. the non-adiabatic binding regime. Here, we introduce a simple analytical framework, known as a piecewise deterministic Markov process (PDMP), that accurately describes the stochastic dynamics of gene expression in the non-adiabatic regime. We illustrate the utility of the PDMP on a non-trivial dynamical system by analysing the properties of a titration-based oscillator in the non-adiabatic limit. We first show how to transform the underlying chemical master equation into a PDMP where the slow transitions between promoter states are stochastic, but whose rates depend upon the faster deterministic dynamics of the transcription factors regulated by these promoters. We show that the PDMP accurately describes the observed periods of stochastic cycles in activator and repressor-based titration oscillators. We then generalize our PDMP analysis to more complicated versions of titration-based oscillators to explain how multiple binding sites lengthen the period and improve coherence. Last, we show how noise-induced oscillation previously observed in a titration-based oscillator arises from non-adiabatic and discrete binding events at the promoter site. PMID:29386401

Detection of long term persistence in time series of the Neuquen River (Argentina)

NASA Astrophysics Data System (ADS)

Seoane, Rafael; Paz González, Antonio

2014-05-01

In the Patagonian region (Argentina), previous hydrometeorological studies that have been developed using general circulation models show variations in annual mean flows. Future climate scenarios obtained from high-resolution models indicate decreases in total annual precipitation, and these scenarios are more important in the Neuquén river basin (23000 km2). The aim of this study was the estimation of long term persistence in the Neuquén River basin (Argentina). The detection of variations in the long range dependence term and long memory of time series was evaluated with the Hurst exponent. We applied rescaled adjusted range analysis (R/S) to time series of River discharges measured from 1903 to 2011 and this time series was divided into two subperiods: the first was from 1903 to 1970 and the second from 1970 to 2011. Results show a small increase in persistence for the second period. Our results are consistent with those obtained by Koch and Markovic (2007), who observed and estimated an increase of the H exponent for the period 1960-2000 in the Elbe River (Germany). References Hurst, H. (1951).Long term storage capacities of reservoirs". Trans. Am. Soc. Civil Engrs., 116:776-808. Koch and Markovic (2007). Evidences for Climate Change in Germany over the 20th Century from the Stochastic Analysis of hydro-meteorological Time Series, MODSIM07, International Congress on Modelling and Simulation, Christchurch, New Zealand.

Driving style recognition method using braking characteristics based on hidden Markov model

PubMed Central

Wu, Chaozhong; Lyu, Nengchao; Huang, Zhen

2017-01-01

Since the advantage of hidden Markov model in dealing with time series data and for the sake of identifying driving style, three driving style (aggressive, moderate and mild) are modeled reasonably through hidden Markov model based on driver braking characteristics to achieve efficient driving style. Firstly, braking impulse and the maximum braking unit area of vacuum booster within a certain time are collected from braking operation, and then general braking and emergency braking characteristics are extracted to code the braking characteristics. Secondly, the braking behavior observation sequence is used to describe the initial parameters of hidden Markov model, and the generation of the hidden Markov model for differentiating and an observation sequence which is trained and judged by the driving style is introduced. Thirdly, the maximum likelihood logarithm could be implied from the observable parameters. The recognition accuracy of algorithm is verified through experiments and two common pattern recognition algorithms. The results showed that the driving style discrimination based on hidden Markov model algorithm could realize effective discriminant of driving style. PMID:28837580

Observation uncertainty in reversible Markov chains.

PubMed

Metzner, Philipp; Weber, Marcus; Schütte, Christof

2010-09-01

In many applications one is interested in finding a simplified model which captures the essential dynamical behavior of a real life process. If the essential dynamics can be assumed to be (approximately) memoryless then a reasonable choice for a model is a Markov model whose parameters are estimated by means of Bayesian inference from an observed time series. We propose an efficient Monte Carlo Markov chain framework to assess the uncertainty of the Markov model and related observables. The derived Gibbs sampler allows for sampling distributions of transition matrices subject to reversibility and/or sparsity constraints. The performance of the suggested sampling scheme is demonstrated and discussed for a variety of model examples. The uncertainty analysis of functions of the Markov model under investigation is discussed in application to the identification of conformations of the trialanine molecule via Robust Perron Cluster Analysis (PCCA+) .

Hydrogeophysical Assessment of Aquifer Uncertainty Using Simulated Annealing driven MRF-Based Stochastic Joint Inversion

NASA Astrophysics Data System (ADS)

Oware, E. K.

2017-12-01

Geophysical quantification of hydrogeological parameters typically involve limited noisy measurements coupled with inadequate understanding of the target phenomenon. Hence, a deterministic solution is unrealistic in light of the largely uncertain inputs. Stochastic imaging (SI), in contrast, provides multiple equiprobable realizations that enable probabilistic assessment of aquifer properties in a realistic manner. Generation of geologically realistic prior models is central to SI frameworks. Higher-order statistics for representing prior geological features in SI are, however, usually borrowed from training images (TIs), which may produce undesirable outcomes if the TIs are unpresentatitve of the target structures. The Markov random field (MRF)-based SI strategy provides a data-driven alternative to TI-based SI algorithms. In the MRF-based method, the simulation of spatial features is guided by Gibbs energy (GE) minimization. Local configurations with smaller GEs have higher likelihood of occurrence and vice versa. The parameters of the Gibbs distribution for computing the GE are estimated from the hydrogeophysical data, thereby enabling the generation of site-specific structures in the absence of reliable TIs. In Metropolis-like SI methods, the variance of the transition probability controls the jump-size. The procedure is a standard Markov chain Monte Carlo (McMC) method when a constant variance is assumed, and becomes simulated annealing (SA) when the variance (cooling temperature) is allowed to decrease gradually with time. We observe that in certain problems, the large variance typically employed at the beginning to hasten burn-in may be unideal for sampling at the equilibrium state. The powerfulness of SA stems from its flexibility to adaptively scale the variance at different stages of the sampling. Degeneration of results were reported in a previous implementation of the MRF-based SI strategy based on a constant variance. Here, we present an updated version of the algorithm based on SA that appears to resolve the degeneration problem with seemingly improved results. We illustrate the performance of the SA version with a joint inversion of time-lapse concentration and electrical resistivity measurements in a hypothetical trinary hydrofacies aquifer characterization problem.

Cluster-based control of a separating flow over a smoothly contoured ramp

NASA Astrophysics Data System (ADS)

Kaiser, Eurika; Noack, Bernd R.; Spohn, Andreas; Cattafesta, Louis N.; Morzyński, Marek

2017-12-01

The ability to manipulate and control fluid flows is of great importance in many scientific and engineering applications. The proposed closed-loop control framework addresses a key issue of model-based control: The actuation effect often results from slow dynamics of strongly nonlinear interactions which the flow reveals at timescales much longer than the prediction horizon of any model. Hence, we employ a probabilistic approach based on a cluster-based discretization of the Liouville equation for the evolution of the probability distribution. The proposed methodology frames high-dimensional, nonlinear dynamics into low-dimensional, probabilistic, linear dynamics which considerably simplifies the optimal control problem while preserving nonlinear actuation mechanisms. The data-driven approach builds upon a state space discretization using a clustering algorithm which groups kinematically similar flow states into a low number of clusters. The temporal evolution of the probability distribution on this set of clusters is then described by a control-dependent Markov model. This Markov model can be used as predictor for the ergodic probability distribution for a particular control law. This probability distribution approximates the long-term behavior of the original system on which basis the optimal control law is determined. We examine how the approach can be used to improve the open-loop actuation in a separating flow dominated by Kelvin-Helmholtz shedding. For this purpose, the feature space, in which the model is learned, and the admissible control inputs are tailored to strongly oscillatory flows.

Gene expression based mouse brain parcellation using Markov random field regularized non-negative matrix factorization

NASA Astrophysics Data System (ADS)

Pathak, Sayan D.; Haynor, David R.; Thompson, Carol L.; Lein, Ed; Hawrylycz, Michael

2009-02-01

Understanding the geography of genetic expression in the mouse brain has opened previously unexplored avenues in neuroinformatics. The Allen Brain Atlas (www.brain-map.org) (ABA) provides genome-wide colorimetric in situ hybridization (ISH) gene expression images at high spatial resolution, all mapped to a common three-dimensional 200μm3 spatial framework defined by the Allen Reference Atlas (ARA) and is a unique data set for studying expression based structural and functional organization of the brain. The goal of this study was to facilitate an unbiased data-driven structural partitioning of the major structures in the mouse brain. We have developed an algorithm that uses nonnegative matrix factorization (NMF) to perform parts based analysis of ISH gene expression images. The standard NMF approach and its variants are limited in their ability to flexibly integrate prior knowledge, in the context of spatial data. In this paper, we introduce spatial connectivity as an additional regularization in NMF decomposition via the use of Markov Random Fields (mNMF). The mNMF algorithm alternates neighborhood updates with iterations of the standard NMF algorithm to exploit spatial correlations in the data. We present the algorithm and show the sub-divisions of hippocampus and somatosensory-cortex obtained via this approach. The results are compared with established neuroanatomic knowledge. We also highlight novel gene expression based sub divisions of the hippocampus identified by using the mNMF algorithm.

Markov models of genome segmentation

NASA Astrophysics Data System (ADS)

Thakur, Vivek; Azad, Rajeev K.; Ramaswamy, Ram

2007-01-01

We introduce Markov models for segmentation of symbolic sequences, extending a segmentation procedure based on the Jensen-Shannon divergence that has been introduced earlier. Higher-order Markov models are more sensitive to the details of local patterns and in application to genome analysis, this makes it possible to segment a sequence at positions that are biologically meaningful. We show the advantage of higher-order Markov-model-based segmentation procedures in detecting compositional inhomogeneity in chimeric DNA sequences constructed from genomes of diverse species, and in application to the E. coli K12 genome, boundaries of genomic islands, cryptic prophages, and horizontally acquired regions are accurately identified.

Wavelet analysis in ecology and epidemiology: impact of statistical tests

PubMed Central

Cazelles, Bernard; Cazelles, Kévin; Chavez, Mario

2014-01-01

Wavelet analysis is now frequently used to extract information from ecological and epidemiological time series. Statistical hypothesis tests are conducted on associated wavelet quantities to assess the likelihood that they are due to a random process. Such random processes represent null models and are generally based on synthetic data that share some statistical characteristics with the original time series. This allows the comparison of null statistics with those obtained from original time series. When creating synthetic datasets, different techniques of resampling result in different characteristics shared by the synthetic time series. Therefore, it becomes crucial to consider the impact of the resampling method on the results. We have addressed this point by comparing seven different statistical testing methods applied with different real and simulated data. Our results show that statistical assessment of periodic patterns is strongly affected by the choice of the resampling method, so two different resampling techniques could lead to two different conclusions about the same time series. Moreover, our results clearly show the inadequacy of resampling series generated by white noise and red noise that are nevertheless the methods currently used in the wide majority of wavelets applications. Our results highlight that the characteristics of a time series, namely its Fourier spectrum and autocorrelation, are important to consider when choosing the resampling technique. Results suggest that data-driven resampling methods should be used such as the hidden Markov model algorithm and the ‘beta-surrogate’ method. PMID:24284892

Wavelet analysis in ecology and epidemiology: impact of statistical tests.

PubMed

Cazelles, Bernard; Cazelles, Kévin; Chavez, Mario

2014-02-06

Wavelet analysis is now frequently used to extract information from ecological and epidemiological time series. Statistical hypothesis tests are conducted on associated wavelet quantities to assess the likelihood that they are due to a random process. Such random processes represent null models and are generally based on synthetic data that share some statistical characteristics with the original time series. This allows the comparison of null statistics with those obtained from original time series. When creating synthetic datasets, different techniques of resampling result in different characteristics shared by the synthetic time series. Therefore, it becomes crucial to consider the impact of the resampling method on the results. We have addressed this point by comparing seven different statistical testing methods applied with different real and simulated data. Our results show that statistical assessment of periodic patterns is strongly affected by the choice of the resampling method, so two different resampling techniques could lead to two different conclusions about the same time series. Moreover, our results clearly show the inadequacy of resampling series generated by white noise and red noise that are nevertheless the methods currently used in the wide majority of wavelets applications. Our results highlight that the characteristics of a time series, namely its Fourier spectrum and autocorrelation, are important to consider when choosing the resampling technique. Results suggest that data-driven resampling methods should be used such as the hidden Markov model algorithm and the 'beta-surrogate' method.

Optimal choice of word length when comparing two Markov sequences using a χ 2-statistic.

PubMed

Bai, Xin; Tang, Kujin; Ren, Jie; Waterman, Michael; Sun, Fengzhu

2017-10-03

Alignment-free sequence comparison using counts of word patterns (grams, k-tuples) has become an active research topic due to the large amount of sequence data from the new sequencing technologies. Genome sequences are frequently modelled by Markov chains and the likelihood ratio test or the corresponding approximate χ 2 -statistic has been suggested to compare two sequences. However, it is not known how to best choose the word length k in such studies. We develop an optimal strategy to choose k by maximizing the statistical power of detecting differences between two sequences. Let the orders of the Markov chains for the two sequences be r 1 and r 2 , respectively. We show through both simulations and theoretical studies that the optimal k= max(r 1 ,r 2 )+1 for both long sequences and next generation sequencing (NGS) read data. The orders of the Markov chains may be unknown and several methods have been developed to estimate the orders of Markov chains based on both long sequences and NGS reads. We study the power loss of the statistics when the estimated orders are used. It is shown that the power loss is minimal for some of the estimators of the orders of Markov chains. Our studies provide guidelines on choosing the optimal word length for the comparison of Markov sequences.

Mesoscopic fluctuations in biharmonically driven flux qubits

NASA Astrophysics Data System (ADS)

Ferrón, Alejandro; Domínguez, Daniel; Sánchez, María José

2017-01-01

We investigate flux qubits driven by a biharmonic magnetic signal, with a phase lag that acts as an effective time reversal broken parameter. The driving induced transition rate between the ground and the excited state of the flux qubit can be thought of as an effective transmittance, profiting from a direct analogy between interference effects at avoided level crossings and scattering events in disordered electronic systems. For time scales prior to full relaxation, but large compared to the decoherence time, this characteristic rate has been accessed experimentally by Gustavsson et al. [Phys. Rev. Lett. 110, 016603 (2013)], 10.1103/PhysRevLett.110.016603 and its sensitivity with both the phase lag and the dc flux detuning explored. In this way, signatures of universal conductance fluctuationslike effects have been analyzed and compared with predictions from a phenomenological model that only accounts for decoherence, as a classical noise. Here we go beyond the classical noise model and solve the full dynamics of the driven flux qubit in contact with a quantum bath employing the Floquet-Born-Markov master equation. Within this formalism, the computed relaxation and decoherence rates turn out to be strongly dependent on both the phase lag and the dc flux detuning. Consequently, the associated pattern of fluctuations in the characteristic rates display important differences with those obtained within the mentioned phenomenological model. In particular, we demonstrate the weak localizationlike effect in the average values of the relaxation rate. Our predictions can be tested for accessible but longer time scales than the current experimental times.

Fast-slow asymptotics for a Markov chain model of fast sodium current

NASA Astrophysics Data System (ADS)

Starý, Tomáš; Biktashev, Vadim N.

2017-09-01

We explore the feasibility of using fast-slow asymptotics to eliminate the computational stiffness of discrete-state, continuous-time deterministic Markov chain models of ionic channels underlying cardiac excitability. We focus on a Markov chain model of fast sodium current, and investigate its asymptotic behaviour with respect to small parameters identified in different ways.

Periodic and quasiperiodic revivals in periodically driven interacting quantum systems

NASA Astrophysics Data System (ADS)

Luitz, David J.; Lazarides, Achilleas; Bar Lev, Yevgeny

2018-01-01

Recently it has been shown that interparticle interactions generically destroy dynamical localization in periodically driven systems, resulting in diffusive transport and heating. In this Rapid Communication we rigorously construct a family of interacting driven systems which are dynamically localized and effectively decoupled from the external driving potential. We show that these systems exhibit tunable periodic or quasiperiodic revivals of the many-body wave function and thus of all physical observables. By numerically examining spinless fermions on a one-dimensional lattice we show that the analytically obtained revivals of such systems remain stable for finite systems with open boundary conditions while having a finite lifetime in the presence of static spatial disorder. We find this lifetime to be inversely proportional to the disorder strength.

Modelisation de l'historique d'operation de groupes turbine-alternateur

NASA Astrophysics Data System (ADS)

Szczota, Mickael

Because of their ageing fleet, the utility managers are increasingly in needs of tools that can help them to plan efficiently maintenance operations. Hydro-Quebec started a project that aim to foresee the degradation of their hydroelectric runner, and use that information to classify the generating unit. That classification will help to know which generating unit is more at risk to undergo a major failure. Cracks linked to the fatigue phenomenon are a predominant degradation mode and the loading sequences applied to the runner is a parameter impacting the crack growth. So, the aim of this memoir is to create a generator able to generate synthetic loading sequences that are statistically equivalent to the observed history. Those simulated sequences will be used as input in a life assessment model. At first, we describe how the generating units are operated by Hydro-Quebec and analyse the available data, the analysis shows that the data are non-stationnary. Then, we review modelisation and validation methods. In the following chapter a particular attention is given to a precise description of the validation and comparison procedure. Then, we present the comparison of three kind of model : Discrete Time Markov Chains, Discrete Time Semi-Markov Chains and the Moving Block Bootstrap. For the first two models, we describe how to take account for the non-stationnarity. Finally, we show that the Markov Chain is not adapted for our case, and that the Semi-Markov chains are better when they include the non-stationnarity. The final choice between Semi-Markov Chains and the Moving Block Bootstrap depends of the user. But, with a long term vision we recommend the use of Semi-Markov chains for their flexibility. Keywords: Stochastic models, Models validation, Reliability, Semi-Markov Chains, Markov Chains, Bootstrap

Saccade selection when reward probability is dynamically manipulated using Markov chains

PubMed Central

Lovejoy, Lee P.; Krauzlis, Richard J.

2012-01-01

Markov chains (stochastic processes where probabilities are assigned based on the previous outcome) are commonly used to examine the transitions between behavioral states, such as those that occur during foraging or social interactions. However, relatively little is known about how well primates can incorporate knowledge about Markov chains into their behavior. Saccadic eye movements are an example of a simple behavior influenced by information about probability, and thus are good candidates for testing whether subjects can learn Markov chains. In addition, when investigating the influence of probability on saccade target selection, the use of Markov chains could provide an alternative method that avoids confounds present in other task designs. To investigate these possibilities, we evaluated human behavior on a task in which stimulus reward probabilities were assigned using a Markov chain. On each trial, the subject selected one of four identical stimuli by saccade; after selection, feedback indicated the rewarded stimulus. Each session consisted of 200–600 trials, and on some sessions, the reward magnitude varied. On sessions with a uniform reward, subjects (n = 6) learned to select stimuli at a frequency close to reward probability, which is similar to human behavior on matching or probability classification tasks. When informed that a Markov chain assigned reward probabilities, subjects (n = 3) learned to select the greatest reward probability more often, bringing them close to behavior that maximizes reward. On sessions where reward magnitude varied across stimuli, subjects (n = 6) demonstrated preferences for both greater reward probability and greater reward magnitude, resulting in a preference for greater expected value (the product of reward probability and magnitude). These results demonstrate that Markov chains can be used to dynamically assign probabilities that are rapidly exploited by human subjects during saccade target selection. PMID:18330552

Saccade selection when reward probability is dynamically manipulated using Markov chains.

PubMed

Nummela, Samuel U; Lovejoy, Lee P; Krauzlis, Richard J

2008-05-01

Markov chains (stochastic processes where probabilities are assigned based on the previous outcome) are commonly used to examine the transitions between behavioral states, such as those that occur during foraging or social interactions. However, relatively little is known about how well primates can incorporate knowledge about Markov chains into their behavior. Saccadic eye movements are an example of a simple behavior influenced by information about probability, and thus are good candidates for testing whether subjects can learn Markov chains. In addition, when investigating the influence of probability on saccade target selection, the use of Markov chains could provide an alternative method that avoids confounds present in other task designs. To investigate these possibilities, we evaluated human behavior on a task in which stimulus reward probabilities were assigned using a Markov chain. On each trial, the subject selected one of four identical stimuli by saccade; after selection, feedback indicated the rewarded stimulus. Each session consisted of 200-600 trials, and on some sessions, the reward magnitude varied. On sessions with a uniform reward, subjects (n = 6) learned to select stimuli at a frequency close to reward probability, which is similar to human behavior on matching or probability classification tasks. When informed that a Markov chain assigned reward probabilities, subjects (n = 3) learned to select the greatest reward probability more often, bringing them close to behavior that maximizes reward. On sessions where reward magnitude varied across stimuli, subjects (n = 6) demonstrated preferences for both greater reward probability and greater reward magnitude, resulting in a preference for greater expected value (the product of reward probability and magnitude). These results demonstrate that Markov chains can be used to dynamically assign probabilities that are rapidly exploited by human subjects during saccade target selection.

Classification of customer lifetime value models using Markov chain

NASA Astrophysics Data System (ADS)

Permana, Dony; Pasaribu, Udjianna S.; Indratno, Sapto W.; Suprayogi

2017-10-01

A firm’s potential reward in future time from a customer can be determined by customer lifetime value (CLV). There are some mathematic methods to calculate it. One method is using Markov chain stochastic model. Here, a customer is assumed through some states. Transition inter the states follow Markovian properties. If we are given some states for a customer and the relationships inter states, then we can make some Markov models to describe the properties of the customer. As Markov models, CLV is defined as a vector contains CLV for a customer in the first state. In this paper we make a classification of Markov Models to calculate CLV. Start from two states of customer model, we make develop in many states models. The development a model is based on weaknesses in previous model. Some last models can be expected to describe how real characters of customers in a firm.

Metrics for Labeled Markov Systems

NASA Technical Reports Server (NTRS)

Desharnais, Josee; Jagadeesan, Radha; Gupta, Vineet; Panangaden, Prakash

1999-01-01

Partial Labeled Markov Chains are simultaneously generalizations of process algebra and of traditional Markov chains. They provide a foundation for interacting discrete probabilistic systems, the interaction being synchronization on labels as in process algebra. Existing notions of process equivalence are too sensitive to the exact probabilities of various transitions. This paper addresses contextual reasoning principles for reasoning about more robust notions of "approximate" equivalence between concurrent interacting probabilistic systems. The present results indicate that:We develop a family of metrics between partial labeled Markov chains to formalize the notion of distance between processes. We show that processes at distance zero are bisimilar. We describe a decision procedure to compute the distance between two processes. We show that reasoning about approximate equivalence can be done compositionally by showing that process combinators do not increase distance. We introduce an asymptotic metric to capture asymptotic properties of Markov chains; and show that parallel composition does not increase asymptotic distance.

Anderson localization and Mott insulator phase in the time domain

PubMed Central

Sacha, Krzysztof

2015-01-01

Particles in space periodic potentials constitute standard models for investigation of crystalline phenomena in solid state physics. Time periodicity of periodically driven systems is a close analogue of space periodicity of solid state crystals. There is an intriguing question if solid state phenomena can be observed in the time domain. Here we show that wave-packets localized on resonant classical trajectories of periodically driven systems are ideal elements to realize Anderson localization or Mott insulator phase in the time domain. Uniform superpositions of the wave-packets form stationary states of a periodically driven particle. However, an additional perturbation that fluctuates in time results in disorder in time and Anderson localization effects emerge. Switching to many-particle systems we observe that depending on how strong particle interactions are, stationary states can be Bose-Einstein condensates or single Fock states where definite numbers of particles occupy the periodically evolving wave-packets. Our study shows that non-trivial crystal-like phenomena can be observed in the time domain. PMID:26074169

Nonequilibrium steady states and resonant tunneling in time-periodically driven systems with interactions

NASA Astrophysics Data System (ADS)

Qin, Tao; Hofstetter, Walter

2018-03-01

Time-periodically driven systems are a versatile toolbox for realizing interesting effective Hamiltonians. Heating, caused by excitations to high-energy states, is a challenge for experiments. While most setups so far address the relatively weakly interacting regime, it is of general interest to study heating in strongly correlated systems. Using Floquet dynamical mean-field theory, we study nonequilibrium steady states (NESS) in the Falicov-Kimball model, with time-periodically driven kinetic energy or interaction. We systematically investigate the nonequilibrium properties of the NESS. For a driven kinetic energy, we show that resonant tunneling, where the interaction is an integer multiple of the driving frequency, plays an important role in the heating. In the strongly correlated regime, we show that this can be well understood using Fermi's golden rule and the Schrieffer-Wolff transformation for a time-periodically driven system. We furthermore demonstrate that resonant tunneling can be used to control the population of Floquet states to achieve "photodoping." For driven interactions introduced by an oscillating magnetic field near a widely adopted Feshbach resonance, we find that the double occupancy is strongly modulated. Our calculations apply to shaken ultracold-atom systems and to solid-state systems in a spatially uniform but time-dependent electric field. They are also closely related to lattice modulation spectroscopy. Our calculations are helpful to understand the latest experiments on strongly correlated Floquet systems.

A fast Karhunen-Loeve transform for a class of random processes

NASA Technical Reports Server (NTRS)

Jain, A. K.

1976-01-01

It is shown that for a class of finite first-order Markov signals, the Karhunen-Loeve (KL) transform for data compression is a set of periodic sine functions if the boundary values of the signal are fixed or known. These sine functions are shown to be related to the Fourier transform so that a fast Fourier transform algorithm can be used to implement the KL transform. Extension to two dimensions with reference to images with separable contravariance function is shown.

Optimal dynamic control of resources in a distributed system

NASA Technical Reports Server (NTRS)

Shin, Kang G.; Krishna, C. M.; Lee, Yann-Hang

1989-01-01

The authors quantitatively formulate the problem of controlling resources in a distributed system so as to optimize a reward function and derive optimal control strategies using Markov decision theory. The control variables treated are quite general; they could be control decisions related to system configuration, repair, diagnostics, files, or data. Two algorithms for resource control in distributed systems are derived for time-invariant and periodic environments, respectively. A detailed example to demonstrate the power and usefulness of the approach is provided.

Educational Aspirations: Markov and Poisson Models. Rural Industrial Development Project Working Paper Number 14, August 1971.

ERIC Educational Resources Information Center

Kayser, Brian D.

The fit of educational aspirations of Illinois rural high school youths to 3 related one-parameter mathematical models was investigated. The models used were the continuous-time Markov chain model, the discrete-time Markov chain, and the Poisson distribution. The sample of 635 students responded to questionnaires from 1966 to 1969 as part of an…

The spectral method and the central limit theorem for general Markov chains

NASA Astrophysics Data System (ADS)

Nagaev, S. V.

2017-12-01

We consider Markov chains with an arbitrary phase space and develop a modification of the spectral method that enables us to prove the central limit theorem (CLT) for non-uniformly ergodic Markov chains. The conditions imposed on the transition function are more general than those by Athreya-Ney and Nummelin. Our proof of the CLT is purely analytical.

SMERFS: Stochastic Markov Evaluation of Random Fields on the Sphere

NASA Astrophysics Data System (ADS)

Creasey, Peter; Lang, Annika

2018-04-01

SMERFS (Stochastic Markov Evaluation of Random Fields on the Sphere) creates large realizations of random fields on the sphere. It uses a fast algorithm based on Markov properties and fast Fourier Transforms in 1d that generates samples on an n X n grid in O(n2 log n) and efficiently derives the necessary conditional covariance matrices.

Markov chains and semi-Markov models in time-to-event analysis.

PubMed

Abner, Erin L; Charnigo, Richard J; Kryscio, Richard J

2013-10-25

A variety of statistical methods are available to investigators for analysis of time-to-event data, often referred to as survival analysis. Kaplan-Meier estimation and Cox proportional hazards regression are commonly employed tools but are not appropriate for all studies, particularly in the presence of competing risks and when multiple or recurrent outcomes are of interest. Markov chain models can accommodate censored data, competing risks (informative censoring), multiple outcomes, recurrent outcomes, frailty, and non-constant survival probabilities. Markov chain models, though often overlooked by investigators in time-to-event analysis, have long been used in clinical studies and have widespread application in other fields.

Markov chains and semi-Markov models in time-to-event analysis

PubMed Central

Abner, Erin L.; Charnigo, Richard J.; Kryscio, Richard J.

2014-01-01

A variety of statistical methods are available to investigators for analysis of time-to-event data, often referred to as survival analysis. Kaplan-Meier estimation and Cox proportional hazards regression are commonly employed tools but are not appropriate for all studies, particularly in the presence of competing risks and when multiple or recurrent outcomes are of interest. Markov chain models can accommodate censored data, competing risks (informative censoring), multiple outcomes, recurrent outcomes, frailty, and non-constant survival probabilities. Markov chain models, though often overlooked by investigators in time-to-event analysis, have long been used in clinical studies and have widespread application in other fields. PMID:24818062

Parsing Social Network Survey Data from Hidden Populations Using Stochastic Context-Free Grammars

PubMed Central

Poon, Art F. Y.; Brouwer, Kimberly C.; Strathdee, Steffanie A.; Firestone-Cruz, Michelle; Lozada, Remedios M.; Kosakovsky Pond, Sergei L.; Heckathorn, Douglas D.; Frost, Simon D. W.

2009-01-01

Background Human populations are structured by social networks, in which individuals tend to form relationships based on shared attributes. Certain attributes that are ambiguous, stigmatized or illegal can create a ÔhiddenÕ population, so-called because its members are difficult to identify. Many hidden populations are also at an elevated risk of exposure to infectious diseases. Consequently, public health agencies are presently adopting modern survey techniques that traverse social networks in hidden populations by soliciting individuals to recruit their peers, e.g., respondent-driven sampling (RDS). The concomitant accumulation of network-based epidemiological data, however, is rapidly outpacing the development of computational methods for analysis. Moreover, current analytical models rely on unrealistic assumptions, e.g., that the traversal of social networks can be modeled by a Markov chain rather than a branching process. Methodology/Principal Findings Here, we develop a new methodology based on stochastic context-free grammars (SCFGs), which are well-suited to modeling tree-like structure of the RDS recruitment process. We apply this methodology to an RDS case study of injection drug users (IDUs) in Tijuana, México, a hidden population at high risk of blood-borne and sexually-transmitted infections (i.e., HIV, hepatitis C virus, syphilis). Survey data were encoded as text strings that were parsed using our custom implementation of the inside-outside algorithm in a publicly-available software package (HyPhy), which uses either expectation maximization or direct optimization methods and permits constraints on model parameters for hypothesis testing. We identified significant latent variability in the recruitment process that violates assumptions of Markov chain-based methods for RDS analysis: firstly, IDUs tended to emulate the recruitment behavior of their own recruiter; and secondly, the recruitment of like peers (homophily) was dependent on the number of recruits. Conclusions SCFGs provide a rich probabilistic language that can articulate complex latent structure in survey data derived from the traversal of social networks. Such structure that has no representation in Markov chain-based models can interfere with the estimation of the composition of hidden populations if left unaccounted for, raising critical implications for the prevention and control of infectious disease epidemics. PMID:19738904

Detecting critical state before phase transition of complex systems by hidden Markov model

NASA Astrophysics Data System (ADS)

Liu, Rui; Chen, Pei; Li, Yongjun; Chen, Luonan

Identifying the critical state or pre-transition state just before the occurrence of a phase transition is a challenging task, because the state of the system may show little apparent change before this critical transition during the gradual parameter variations. Such dynamics of phase transition is generally composed of three stages, i.e., before-transition state, pre-transition state, and after-transition state, which can be considered as three different Markov processes. Thus, based on this dynamical feature, we present a novel computational method, i.e., hidden Markov model (HMM), to detect the switching point of the two Markov processes from the before-transition state (a stationary Markov process) to the pre-transition state (a time-varying Markov process), thereby identifying the pre-transition state or early-warning signals of the phase transition. To validate the effectiveness, we apply this method to detect the signals of the imminent phase transitions of complex systems based on the simulated datasets, and further identify the pre-transition states as well as their critical modules for three real datasets, i.e., the acute lung injury triggered by phosgene inhalation, MCF-7 human breast cancer caused by heregulin, and HCV-induced dysplasia and hepatocellular carcinoma.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Costa, Liborio I., E-mail: liborio78@gmail.com

A new Markov Chain Monte Carlo method for simulating the dynamics of particle systems characterized by hard-core interactions is introduced. In contrast to traditional Kinetic Monte Carlo approaches, where the state of the system is associated with minima in the energy landscape, in the proposed method, the state of the system is associated with the set of paths traveled by the atoms and the transition probabilities for an atom to be displaced are proportional to the corresponding velocities. In this way, the number of possible state-to-state transitions is reduced to a discrete set, and a direct link between the Montemore » Carlo time step and true physical time is naturally established. The resulting rejection-free algorithm is validated against event-driven molecular dynamics: the equilibrium and non-equilibrium dynamics of hard disks converge to the exact results with decreasing displacement size.« less

Markov and semi-Markov switching linear mixed models used to identify forest tree growth components.

PubMed

Chaubert-Pereira, Florence; Guédon, Yann; Lavergne, Christian; Trottier, Catherine

2010-09-01

Tree growth is assumed to be mainly the result of three components: (i) an endogenous component assumed to be structured as a succession of roughly stationary phases separated by marked change points that are asynchronous among individuals, (ii) a time-varying environmental component assumed to take the form of synchronous fluctuations among individuals, and (iii) an individual component corresponding mainly to the local environment of each tree. To identify and characterize these three components, we propose to use semi-Markov switching linear mixed models, i.e., models that combine linear mixed models in a semi-Markovian manner. The underlying semi-Markov chain represents the succession of growth phases and their lengths (endogenous component) whereas the linear mixed models attached to each state of the underlying semi-Markov chain represent-in the corresponding growth phase-both the influence of time-varying climatic covariates (environmental component) as fixed effects, and interindividual heterogeneity (individual component) as random effects. In this article, we address the estimation of Markov and semi-Markov switching linear mixed models in a general framework. We propose a Monte Carlo expectation-maximization like algorithm whose iterations decompose into three steps: (i) sampling of state sequences given random effects, (ii) prediction of random effects given state sequences, and (iii) maximization. The proposed statistical modeling approach is illustrated by the analysis of successive annual shoots along Corsican pine trunks influenced by climatic covariates. © 2009, The International Biometric Society.

Strategic level proton therapy patient admission planning: a Markov decision process modeling approach.

PubMed

Gedik, Ridvan; Zhang, Shengfan; Rainwater, Chase

2017-06-01

A relatively new consideration in proton therapy planning is the requirement that the mix of patients treated from different categories satisfy desired mix percentages. Deviations from these percentages and their impacts on operational capabilities are of particular interest to healthcare planners. In this study, we investigate intelligent ways of admitting patients to a proton therapy facility that maximize the total expected number of treatment sessions (fractions) delivered to patients in a planning period with stochastic patient arrivals and penalize the deviation from the patient mix restrictions. We propose a Markov Decision Process (MDP) model that provides very useful insights in determining the best patient admission policies in the case of an unexpected opening in the facility (i.e., no-shows, appointment cancellations, etc.). In order to overcome the curse of dimensionality for larger and more realistic instances, we propose an aggregate MDP model that is able to approximate optimal patient admission policies using the worded weight aggregation technique. Our models are applicable to healthcare treatment facilities throughout the United States, but are motivated by collaboration with the University of Florida Proton Therapy Institute (UFPTI).

Urban change analysis and future growth of Istanbul.

PubMed

Akın, Anıl; Sunar, Filiz; Berberoğlu, Süha

2015-08-01

This study is aimed at analyzing urban change within Istanbul and assessing the city's future growth potential using appropriate approach modeling for the year 2040. Urban growth is a major driving force of land-use change, and spatial and temporal components of urbanization can be identified through accurate spatial modeling. In this context, widely used urban modeling approaches, such as the Markov chain and logistic regression based on cellular automata (CA), were used to simulate urban growth within Istanbul. The distance from each pixel to the urban and road classes, elevation, and slope, together with municipality and land use maps (as an excluded layer), were identified as factors. Calibration data were obtained from remotely sensed data recorded in 1972, 1986, and 2013. Validation was performed by overlaying the simulated and actual 2013 urban maps, and a kappa index of agreement was derived. The results indicate that urban expansion will influence mainly forest areas during the time period of 2013-2040. The urban expansion was predicted as 429 and 327 km(2) with the Markov chain and logistic regression models, respectively.

Exceptional motifs in different Markov chain models for a statistical analysis of DNA sequences.

PubMed

Schbath, S; Prum, B; de Turckheim, E

1995-01-01

Identifying exceptional motifs is often used for extracting information from long DNA sequences. The two difficulties of the method are the choice of the model that defines the expected frequencies of words and the approximation of the variance of the difference T(W) between the number of occurrences of a word W and its estimation. We consider here different Markov chain models, either with stationary or periodic transition probabilities. We estimate the variance of the difference T(W) by the conditional variance of the number of occurrences of W given the oligonucleotides counts that define the model. Two applications show how to use asymptotically standard normal statistics associated with the counts to describe a given sequence in terms of its outlying words. Sequences of Escherichia coli and of Bacillus subtilis are compared with respect to their exceptional tri- and tetranucleotides. For both bacteria, exceptional 3-words are mainly found in the coding frame. E. coli palindrome counts are analyzed in different models, showing that many overabundant words are one-letter mutations of avoided palindromes.

Theory of many-body localization in periodically driven systems

DOE Office of Scientific and Technical Information (OSTI.GOV)

Abanin, Dmitry A., E-mail: dabanin@gmail.com; De Roeck, Wojciech; Huveneers, François

We present a theory of periodically driven, many-body localized (MBL) systems. We argue that MBL persists under periodic driving at high enough driving frequency: The Floquet operator (evolution operator over one driving period) can be represented as an exponential of an effective time-independent Hamiltonian, which is a sum of quasi-local terms and is itself fully MBL. We derive this result by constructing a sequence of canonical transformations to remove the time-dependence from the original Hamiltonian. When the driving evolves smoothly in time, the theory can be sharpened by estimating the probability of adiabatic Landau–Zener transitions at many-body level crossings. Inmore » all cases, we argue that there is delocalization at sufficiently low frequency. We propose a phase diagram of driven MBL systems.« less

Discriminative Learning with Markov Logic Networks

DTIC Science & Technology

2009-10-01

Discriminative Learning with Markov Logic Networks Tuyen N. Huynh Department of Computer Sciences University of Texas at Austin Austin, TX 78712...emerging area of research that addresses the problem of learning from noisy structured/relational data. Markov logic networks (MLNs), sets of weighted...TASK NUMBER 5f. WORK UNIT NUMBER 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) University of Texas at Austin,Department of Computer

Modeling the hydrological impacts of land use/land cover changes in the Andassa watershed, Blue Nile Basin, Ethiopia.

PubMed

Gashaw, Temesgen; Tulu, Taffa; Argaw, Mekuria; Worqlul, Abeyou W

2018-04-01

Understanding the hydrological response of a watershed to land use/land cover (LULC) changes is imperative for water resources management planning. The objective of this study was to analyze the hydrological impacts of LULC changes in the Andassa watershed for a period of 1985-2015 and to predict the LULC change impact on the hydrological status in year 2045. The hybrid land use classification technique for classifying Landsat images (1985, 2000 and 2015); Cellular-Automata Markov (CA-Markov) for prediction of the 2030 and 2045 LULC states; the Soil and Water Assessment Tool (SWAT) for hydrological modeling were employed in the analyses. In order to isolate the impacts of LULC changes, the LULC maps were used independently while keeping the other SWAT inputs constant. The contribution of each of the LULC classes was examined with the Partial Least Squares Regression (PLSR) model. The results showed that there was a continuous expansion of cultivated land and built-up area, and withdrawing of forest, shrubland and grassland during the 1985-2015 periods, which are expected to continue in the 2030 and 2045 periods. The LULC changes, which had occurred during the period of 1985 to 2015, had increased the annual flow (2.2%), wet seasonal flow (4.6%), surface runoff (9.3%) and water yield (2.4%). Conversely, the observed changes had reduced dry season flow (2.8%), lateral flow (5.7%), groundwater flow (7.8%) and ET (0.3%). The 2030 and 2045 LULC states are expected to further increase the annual and wet season flow, surface runoff and water yield, and reduce dry season flow, groundwater flow, lateral flow and ET. The change in hydrological components is a direct result of the significant transition from the vegetation to non-vegetation cover in the watershed. This suggests an urgent need to regulate the LULC in order to maintain the hydrological balance. Copyright © 2017 Elsevier B.V. All rights reserved.

[Application of Markov model in post-marketing pharmacoeconomic evaluation of traditional Chinese medicine].

PubMed

Wang, Xin; Su, Xia; Sun, Wentao; Xie, Yanming; Wang, Yongyan

2011-10-01

In post-marketing study of traditional Chinese medicine (TCM), pharmacoeconomic evaluation has an important applied significance. However, the economic literatures of TCM have been unable to fully and accurately reflect the unique overall outcomes of treatment with TCM. For the special nature of TCM itself, we recommend that Markov model could be introduced into post-marketing pharmacoeconomic evaluation of TCM, and also explore the feasibility of model application. Markov model can extrapolate the study time horizon, suit with effectiveness indicators of TCM, and provide measurable comprehensive outcome. In addition, Markov model can promote the development of TCM quality of life scale and the methodology of post-marketing pharmacoeconomic evaluation.

On the utility of the multi-level algorithm for the solution of nearly completely decomposable Markov chains

NASA Technical Reports Server (NTRS)

Leutenegger, Scott T.; Horton, Graham

1994-01-01

Recently the Multi-Level algorithm was introduced as a general purpose solver for the solution of steady state Markov chains. In this paper, we consider the performance of the Multi-Level algorithm for solving Nearly Completely Decomposable (NCD) Markov chains, for which special-purpose iteractive aggregation/disaggregation algorithms such as the Koury-McAllister-Stewart (KMS) method have been developed that can exploit the decomposability of the the Markov chain. We present experimental results indicating that the general-purpose Multi-Level algorithm is competitive, and can be significantly faster than the special-purpose KMS algorithm when Gauss-Seidel and Gaussian Elimination are used for solving the individual blocks.

Policy Transfer via Markov Logic Networks

NASA Astrophysics Data System (ADS)

Torrey, Lisa; Shavlik, Jude

We propose using a statistical-relational model, the Markov Logic Network, for knowledge transfer in reinforcement learning. Our goal is to extract relational knowledge from a source task and use it to speed up learning in a related target task. We show that Markov Logic Networks are effective models for capturing both source-task Q-functions and source-task policies. We apply them via demonstration, which involves using them for decision making in an initial stage of the target task before continuing to learn. Through experiments in the RoboCup simulated-soccer domain, we show that transfer via Markov Logic Networks can significantly improve early performance in complex tasks, and that transferring policies is more effective than transferring Q-functions.

Modeling the coupled return-spread high frequency dynamics of large tick assets

NASA Astrophysics Data System (ADS)

Curato, Gianbiagio; Lillo, Fabrizio

2015-01-01

Large tick assets, i.e. assets where one tick movement is a significant fraction of the price and bid-ask spread is almost always equal to one tick, display a dynamics in which price changes and spread are strongly coupled. We present an approach based on the hidden Markov model, also known in econometrics as the Markov switching model, for the dynamics of price changes, where the latent Markov process is described by the transitions between spreads. We then use a finite Markov mixture of logit regressions on past squared price changes to describe temporal dependencies in the dynamics of price changes. The model can thus be seen as a double chain Markov model. We show that the model describes the shape of the price change distribution at different time scales, volatility clustering, and the anomalous decrease of kurtosis. We calibrate our models based on Nasdaq stocks and we show that this model reproduces remarkably well the statistical properties of real data.

Quantum Enhanced Inference in Markov Logic Networks

NASA Astrophysics Data System (ADS)

Wittek, Peter; Gogolin, Christian

2017-04-01

Markov logic networks (MLNs) reconcile two opposing schools in machine learning and artificial intelligence: causal networks, which account for uncertainty extremely well, and first-order logic, which allows for formal deduction. An MLN is essentially a first-order logic template to generate Markov networks. Inference in MLNs is probabilistic and it is often performed by approximate methods such as Markov chain Monte Carlo (MCMC) Gibbs sampling. An MLN has many regular, symmetric structures that can be exploited at both first-order level and in the generated Markov network. We analyze the graph structures that are produced by various lifting methods and investigate the extent to which quantum protocols can be used to speed up Gibbs sampling with state preparation and measurement schemes. We review different such approaches, discuss their advantages, theoretical limitations, and their appeal to implementations. We find that a straightforward application of a recent result yields exponential speedup compared to classical heuristics in approximate probabilistic inference, thereby demonstrating another example where advanced quantum resources can potentially prove useful in machine learning.

Generation of intervention strategy for a genetic regulatory network represented by a family of Markov Chains.

PubMed

Berlow, Noah; Pal, Ranadip

2011-01-01

Genetic Regulatory Networks (GRNs) are frequently modeled as Markov Chains providing the transition probabilities of moving from one state of the network to another. The inverse problem of inference of the Markov Chain from noisy and limited experimental data is an ill posed problem and often generates multiple model possibilities instead of a unique one. In this article, we address the issue of intervention in a genetic regulatory network represented by a family of Markov Chains. The purpose of intervention is to alter the steady state probability distribution of the GRN as the steady states are considered to be representative of the phenotypes. We consider robust stationary control policies with best expected behavior. The extreme computational complexity involved in search of robust stationary control policies is mitigated by using a sequential approach to control policy generation and utilizing computationally efficient techniques for updating the stationary probability distribution of a Markov chain following a rank one perturbation.

Simplification of Markov chains with infinite state space and the mathematical theory of random gene expression bursts.

PubMed

Jia, Chen

2017-09-01

Here we develop an effective approach to simplify two-time-scale Markov chains with infinite state spaces by removal of states with fast leaving rates, which improves the simplification method of finite Markov chains. We introduce the concept of fast transition paths and show that the effective transitions of the reduced chain can be represented as the superposition of the direct transitions and the indirect transitions via all the fast transition paths. Furthermore, we apply our simplification approach to the standard Markov model of single-cell stochastic gene expression and provide a mathematical theory of random gene expression bursts. We give the precise mathematical conditions for the bursting kinetics of both mRNAs and proteins. It turns out that random bursts exactly correspond to the fast transition paths of the Markov model. This helps us gain a better understanding of the physics behind the bursting kinetics as an emergent behavior from the fundamental multiscale biochemical reaction kinetics of stochastic gene expression.

H2-control and the separation principle for discrete-time jump systems with the Markov chain in a general state space

NASA Astrophysics Data System (ADS)

Figueiredo, Danilo Zucolli; Costa, Oswaldo Luiz do Valle

2017-10-01

This paper deals with the H2 optimal control problem of discrete-time Markov jump linear systems (MJLS) considering the case in which the Markov chain takes values in a general Borel space ?. It is assumed that the controller has access only to an output variable and to the jump parameter. The goal, in this case, is to design a dynamic Markov jump controller such that the H2-norm of the closed-loop system is minimised. It is shown that the H2-norm can be written as the sum of two H2-norms, such that one of them does not depend on the control, and the other one is obtained from the optimal filter for an infinite-horizon filtering problem. This result can be seen as a separation principle for MJLS with Markov chain in a Borel space ? considering the infinite time horizon case.

Simplification of Markov chains with infinite state space and the mathematical theory of random gene expression bursts

NASA Astrophysics Data System (ADS)

Jia, Chen

2017-09-01

Here we develop an effective approach to simplify two-time-scale Markov chains with infinite state spaces by removal of states with fast leaving rates, which improves the simplification method of finite Markov chains. We introduce the concept of fast transition paths and show that the effective transitions of the reduced chain can be represented as the superposition of the direct transitions and the indirect transitions via all the fast transition paths. Furthermore, we apply our simplification approach to the standard Markov model of single-cell stochastic gene expression and provide a mathematical theory of random gene expression bursts. We give the precise mathematical conditions for the bursting kinetics of both mRNAs and proteins. It turns out that random bursts exactly correspond to the fast transition paths of the Markov model. This helps us gain a better understanding of the physics behind the bursting kinetics as an emergent behavior from the fundamental multiscale biochemical reaction kinetics of stochastic gene expression.

Finding exact constants in a Markov model of Zipfs law generation

NASA Astrophysics Data System (ADS)

Bochkarev, V. V.; Lerner, E. Yu.; Nikiforov, A. A.; Pismenskiy, A. A.

2017-12-01

According to the classical Zipfs law, the word frequency is a power function of the word rank with an exponent -1. The objective of this work is to find multiplicative constant in a Markov model of word generation. Previously, the case of independent letters was mathematically strictly investigated in [Bochkarev V V and Lerner E Yu 2017 International Journal of Mathematics and Mathematical Sciences Article ID 914374]. Unfortunately, the methods used in this paper cannot be generalized in case of Markov chains. The search of the correct formulation of the Markov generalization of this results was performed using experiments with different ergodic matrices of transition probability P. Combinatory technique allowed taking into account all the words with probability of more than e -300 in case of 2 by 2 matrices. It was experimentally proved that the required constant in the limit is equal to the value reciprocal to conditional entropy of matrix row P with weights presenting the elements of the vector π of the stationary distribution of the Markov chain.

Quantum Enhanced Inference in Markov Logic Networks.

PubMed

Wittek, Peter; Gogolin, Christian

2017-04-19

Markov logic networks (MLNs) reconcile two opposing schools in machine learning and artificial intelligence: causal networks, which account for uncertainty extremely well, and first-order logic, which allows for formal deduction. An MLN is essentially a first-order logic template to generate Markov networks. Inference in MLNs is probabilistic and it is often performed by approximate methods such as Markov chain Monte Carlo (MCMC) Gibbs sampling. An MLN has many regular, symmetric structures that can be exploited at both first-order level and in the generated Markov network. We analyze the graph structures that are produced by various lifting methods and investigate the extent to which quantum protocols can be used to speed up Gibbs sampling with state preparation and measurement schemes. We review different such approaches, discuss their advantages, theoretical limitations, and their appeal to implementations. We find that a straightforward application of a recent result yields exponential speedup compared to classical heuristics in approximate probabilistic inference, thereby demonstrating another example where advanced quantum resources can potentially prove useful in machine learning.

Quantum Enhanced Inference in Markov Logic Networks

PubMed Central

Wittek, Peter; Gogolin, Christian

2017-01-01

Markov logic networks (MLNs) reconcile two opposing schools in machine learning and artificial intelligence: causal networks, which account for uncertainty extremely well, and first-order logic, which allows for formal deduction. An MLN is essentially a first-order logic template to generate Markov networks. Inference in MLNs is probabilistic and it is often performed by approximate methods such as Markov chain Monte Carlo (MCMC) Gibbs sampling. An MLN has many regular, symmetric structures that can be exploited at both first-order level and in the generated Markov network. We analyze the graph structures that are produced by various lifting methods and investigate the extent to which quantum protocols can be used to speed up Gibbs sampling with state preparation and measurement schemes. We review different such approaches, discuss their advantages, theoretical limitations, and their appeal to implementations. We find that a straightforward application of a recent result yields exponential speedup compared to classical heuristics in approximate probabilistic inference, thereby demonstrating another example where advanced quantum resources can potentially prove useful in machine learning. PMID:28422093

Handling target obscuration through Markov chain observations

NASA Astrophysics Data System (ADS)

Kouritzin, Michael A.; Wu, Biao

2008-04-01

Target Obscuration, including foliage or building obscuration of ground targets and landscape or horizon obscuration of airborne targets, plagues many real world filtering problems. In particular, ground moving target identification Doppler radar, mounted on a surveillance aircraft or unattended airborne vehicle, is used to detect motion consistent with targets of interest. However, these targets try to obscure themselves (at least partially) by, for example, traveling along the edge of a forest or around buildings. This has the effect of creating random blockages in the Doppler radar image that move dynamically and somewhat randomly through this image. Herein, we address tracking problems with target obscuration by building memory into the observations, eschewing the usual corrupted, distorted partial measurement assumptions of filtering in favor of dynamic Markov chain assumptions. In particular, we assume the observations are a Markov chain whose transition probabilities depend upon the signal. The state of the observation Markov chain attempts to depict the current obscuration and the Markov chain dynamics are used to handle the evolution of the partially obscured radar image. Modifications of the classical filtering equations that allow observation memory (in the form of a Markov chain) are given. We use particle filters to estimate the position of the moving targets. Moreover, positive proof-of-concept simulations are included.

Efficient analysis of stochastic gene dynamics in the non-adiabatic regime using piecewise deterministic Markov processes

DOE Office of Scientific and Technical Information (OSTI.GOV)

Lin, Yen Ting; Buchler, Nicolas E.

Single-cell experiments show that gene expression is stochastic and bursty, a feature that can emerge from slow switching between promoter states with different activities. In addition to slow chromatin and/or DNA looping dynamics, one source of long-lived promoter states is the slow binding and unbinding kinetics of transcription factors to promoters, i.e. the non-adiabatic binding regime. Here, we introduce a simple analytical framework, known as a piecewise deterministic Markov process (PDMP), that accurately describes the stochastic dynamics of gene expression in the non-adiabatic regime. We illustrate the utility of the PDMP on a non-trivial dynamical system by analysing the propertiesmore » of a titration-based oscillator in the non-adiabatic limit. We first show how to transform the underlying chemical master equation into a PDMP where the slow transitions between promoter states are stochastic, but whose rates depend upon the faster deterministic dynamics of the transcription factors regulated by these promoters. We show that the PDMP accurately describes the observed periods of stochastic cycles in activator and repressor-based titration oscillators. We then generalize our PDMP analysis to more complicated versions of titration-based oscillators to explain how multiple binding sites lengthen the period and improve coherence. Finally, we show how noise-induced oscillation previously observed in a titration-based oscillator arises from non-adiabatic and discrete binding events at the promoter site.« less

Efficient analysis of stochastic gene dynamics in the non-adiabatic regime using piecewise deterministic Markov processes

DOE PAGES

Lin, Yen Ting; Buchler, Nicolas E.

2018-01-31

Single-cell experiments show that gene expression is stochastic and bursty, a feature that can emerge from slow switching between promoter states with different activities. In addition to slow chromatin and/or DNA looping dynamics, one source of long-lived promoter states is the slow binding and unbinding kinetics of transcription factors to promoters, i.e. the non-adiabatic binding regime. Here, we introduce a simple analytical framework, known as a piecewise deterministic Markov process (PDMP), that accurately describes the stochastic dynamics of gene expression in the non-adiabatic regime. We illustrate the utility of the PDMP on a non-trivial dynamical system by analysing the propertiesmore » of a titration-based oscillator in the non-adiabatic limit. We first show how to transform the underlying chemical master equation into a PDMP where the slow transitions between promoter states are stochastic, but whose rates depend upon the faster deterministic dynamics of the transcription factors regulated by these promoters. We show that the PDMP accurately describes the observed periods of stochastic cycles in activator and repressor-based titration oscillators. We then generalize our PDMP analysis to more complicated versions of titration-based oscillators to explain how multiple binding sites lengthen the period and improve coherence. Finally, we show how noise-induced oscillation previously observed in a titration-based oscillator arises from non-adiabatic and discrete binding events at the promoter site.« less

Assessing significance in a Markov chain without mixing.

PubMed

Chikina, Maria; Frieze, Alan; Pegden, Wesley

2017-03-14

We present a statistical test to detect that a presented state of a reversible Markov chain was not chosen from a stationary distribution. In particular, given a value function for the states of the Markov chain, we would like to show rigorously that the presented state is an outlier with respect to the values, by establishing a [Formula: see text] value under the null hypothesis that it was chosen from a stationary distribution of the chain. A simple heuristic used in practice is to sample ranks of states from long random trajectories on the Markov chain and compare these with the rank of the presented state; if the presented state is a [Formula: see text] outlier compared with the sampled ranks (its rank is in the bottom [Formula: see text] of sampled ranks), then this observation should correspond to a [Formula: see text] value of [Formula: see text] This significance is not rigorous, however, without good bounds on the mixing time of the Markov chain. Our test is the following: Given the presented state in the Markov chain, take a random walk from the presented state for any number of steps. We prove that observing that the presented state is an [Formula: see text]-outlier on the walk is significant at [Formula: see text] under the null hypothesis that the state was chosen from a stationary distribution. We assume nothing about the Markov chain beyond reversibility and show that significance at [Formula: see text] is best possible in general. We illustrate the use of our test with a potential application to the rigorous detection of gerrymandering in Congressional districting.

Assessing significance in a Markov chain without mixing

PubMed Central

Chikina, Maria; Frieze, Alan; Pegden, Wesley

2017-01-01

We present a statistical test to detect that a presented state of a reversible Markov chain was not chosen from a stationary distribution. In particular, given a value function for the states of the Markov chain, we would like to show rigorously that the presented state is an outlier with respect to the values, by establishing a p value under the null hypothesis that it was chosen from a stationary distribution of the chain. A simple heuristic used in practice is to sample ranks of states from long random trajectories on the Markov chain and compare these with the rank of the presented state; if the presented state is a 0.1% outlier compared with the sampled ranks (its rank is in the bottom 0.1% of sampled ranks), then this observation should correspond to a p value of 0.001. This significance is not rigorous, however, without good bounds on the mixing time of the Markov chain. Our test is the following: Given the presented state in the Markov chain, take a random walk from the presented state for any number of steps. We prove that observing that the presented state is an ε-outlier on the walk is significant at p=2ε under the null hypothesis that the state was chosen from a stationary distribution. We assume nothing about the Markov chain beyond reversibility and show that significance at p≈ε is best possible in general. We illustrate the use of our test with a potential application to the rigorous detection of gerrymandering in Congressional districting. PMID:28246331

Non-Markovian dynamics in chiral quantum networks with spins and photons

NASA Astrophysics Data System (ADS)

Ramos, Tomás; Vermersch, Benoît; Hauke, Philipp; Pichler, Hannes; Zoller, Peter

2016-06-01

We study the dynamics of chiral quantum networks consisting of nodes coupled by unidirectional or asymmetric bidirectional quantum channels. In contrast to familiar photonic networks where driven two-level atoms exchange photons via 1D photonic nanostructures, we propose and study a setup where interactions between the atoms are mediated by spin excitations (magnons) in 1D X X spin chains representing spin waveguides. While Markovian quantum network theory eliminates quantum channels as structureless reservoirs in a Born-Markov approximation to obtain a master equation for the nodes, we are interested in non-Markovian dynamics. This arises from the nonlinear character of the dispersion with band-edge effects, and from finite spin propagation velocities leading to time delays in interactions. To account for the non-Markovian dynamics we treat the quantum degrees of freedom of the nodes and connecting channel as a composite spin system with the surrounding of the quantum network as a Markovian bath, allowing for an efficient solution with time-dependent density matrix renormalization-group techniques. We illustrate our approach showing non-Markovian effects in the driven-dissipative formation of quantum dimers, and we present examples for quantum information protocols involving quantum state transfer with engineered elements as basic building blocks of quantum spintronic circuits.

Critical Dynamics of Gravito-Convective Mixing in Geological Carbon Sequestration

PubMed Central

Soltanian, Mohamad Reza; Amooie, Mohammad Amin; Dai, Zhenxue; Cole, David; Moortgat, Joachim

2016-01-01

When CO2 is injected in saline aquifers, dissolution causes a local increase in brine density that can cause Rayleigh-Taylor-type gravitational instabilities. Depending on the Rayleigh number, density-driven flow may mix dissolved CO2 throughout the aquifer at fast advective time-scales through convective mixing. Heterogeneity can impact density-driven flow to different degrees. Zones with low effective vertical permeability may suppress fingering and reduce vertical spreading, while potentially increasing transverse mixing. In more complex heterogeneity, arising from the spatial organization of sedimentary facies, finger propagation is reduced in low permeability facies, but may be enhanced through more permeable facies. The connectivity of facies is critical in determining the large-scale transport of CO2-rich brine. We perform high-resolution finite element simulations of advection-diffusion transport of CO2 with a focus on facies-based bimodal heterogeneity. Permeability fields are generated by a Markov Chain approach, which represent facies architecture by commonly observed characteristics such as volume fractions. CO2 dissolution and phase behavior are modeled with the cubic-plus-association equation-of-state. Our results show that the organization of high-permeability facies and their connectivity control the dynamics of gravitationally unstable flow. We discover new flow regimes in both homogeneous and heterogeneous media and present quantitative scaling relations for their temporal evolution. PMID:27808178

Data-driven fault detection, isolation and estimation of aircraft gas turbine engine actuator and sensors

NASA Astrophysics Data System (ADS)

Naderi, E.; Khorasani, K.

2018-02-01

In this work, a data-driven fault detection, isolation, and estimation (FDI&E) methodology is proposed and developed specifically for monitoring the aircraft gas turbine engine actuator and sensors. The proposed FDI&E filters are directly constructed by using only the available system I/O data at each operating point of the engine. The healthy gas turbine engine is stimulated by a sinusoidal input containing a limited number of frequencies. First, the associated system Markov parameters are estimated by using the FFT of the input and output signals to obtain the frequency response of the gas turbine engine. These data are then used for direct design and realization of the fault detection, isolation and estimation filters. Our proposed scheme therefore does not require any a priori knowledge of the system linear model or its number of poles and zeros at each operating point. We have investigated the effects of the size of the frequency response data on the performance of our proposed schemes. We have shown through comprehensive case studies simulations that desirable fault detection, isolation and estimation performance metrics defined in terms of the confusion matrix criterion can be achieved by having access to only the frequency response of the system at only a limited number of frequencies.

Electromagnetically induced transparency and Autler-Townes splitting in superconducting flux quantum circuits

NASA Astrophysics Data System (ADS)

Sun, Hui-Chen; Liu, Yu-xi; Ian, Hou; You, J. Q.; Il'ichev, E.; Nori, Franco

2014-06-01

We study the microwave absorption of a driven three-level quantum system, which is realized by a superconducting flux quantum circuit (SFQC), with a magnetic driving field applied to the two upper levels. The interaction between the three-level system and its environment is studied within the Born-Markov approximation, and we take into account the effects of the driving field on the damping rates of the three-level system. We study the linear response of the driven three-level SFQC to a weak probe field. The linear magnetic susceptibility of the SFQC can be changed by both the driving field and the bias magnetic flux. When the bias magnetic flux is at the optimal point, the transition from the ground state to the second-excited state is forbidden and the three-level SFQC has a ladder-type transition. Thus, the SFQC responds to the probe field like natural atoms with ladder-type transitions. However, when the bias magnetic flux deviates from the optimal point, the three-level SFQC has a cyclic transition, thus it responds to the probe field like a combination of natural atoms with ladder-type transitions and natural atoms with Λ-type transitions. In particular, we provide detailed discussions on the conditions for realizing electromagnetically induced transparency and Autler-Townes splitting in three-level SFQCs.

Network Structure and Biased Variance Estimation in Respondent Driven Sampling

PubMed Central

Verdery, Ashton M.; Mouw, Ted; Bauldry, Shawn; Mucha, Peter J.

2015-01-01

This paper explores bias in the estimation of sampling variance in Respondent Driven Sampling (RDS). Prior methodological work on RDS has focused on its problematic assumptions and the biases and inefficiencies of its estimators of the population mean. Nonetheless, researchers have given only slight attention to the topic of estimating sampling variance in RDS, despite the importance of variance estimation for the construction of confidence intervals and hypothesis tests. In this paper, we show that the estimators of RDS sampling variance rely on a critical assumption that the network is First Order Markov (FOM) with respect to the dependent variable of interest. We demonstrate, through intuitive examples, mathematical generalizations, and computational experiments that current RDS variance estimators will always underestimate the population sampling variance of RDS in empirical networks that do not conform to the FOM assumption. Analysis of 215 observed university and school networks from Facebook and Add Health indicates that the FOM assumption is violated in every empirical network we analyze, and that these violations lead to substantially biased RDS estimators of sampling variance. We propose and test two alternative variance estimators that show some promise for reducing biases, but which also illustrate the limits of estimating sampling variance with only partial information on the underlying population social network. PMID:26679927

Conditioned Limit Theorems for Some Null Recurrent Markov Processes

DTIC Science & Technology

1976-08-01

Chapter 1 INTRODUCTION 1.1 Summary of Results Let (Vk, k ! 0) be a discrete time Markov process with state space EC(- , ) and let S be...explain our results in some detail. 2 We begin by stating our three basic assumptions: (1) vk s k 2 0 Is a Markov process with state space E C(-o,%); (Ii... 12 n 3. CONDITIONING ON T (, > n.................................1.9 3.1 Preliminary Results

Filtering Using Nonlinear Expectations

DTIC Science & Technology

2016-04-16

gives a solution to estimating a Markov chain observed in Gaussian noise when the variance of the noise is unkown. This paper is accepted for the IEEE...Optimization, an A* journal. A short third paper discusses how to estimate a change in the transition dynamics of a noisily observed Markov chain ...The change point time is hidden in a hidden Markov chain , so a second level of discovery is involved. This paper is accepted for Communications in

DOE Office of Scientific and Technical Information (OSTI.GOV)

Cai, H.

In this dissertation we study a procedure which restarts a Markov process when the process is killed by some arbitrary multiplicative functional. The regenerative nature of this revival procedure is characterized through a Markov renewal equation. An interesting duality between the revival procedure and the classical killing operation is found. Under the condition that the multiplicative functional possesses an intensity, the generators of the revival process can be written down explicitly. An intimate connection is also found between the perturbation of the sample path of a Markov process and the perturbation of a generator (in Kato's sense). The applications ofmore » the theory include the study of the processes like piecewise-deterministic Markov process, virtual waiting time process and the first entrance decomposition (taboo probability).« less

Markov and non-Markov processes in complex systems by the dynamical information entropy

NASA Astrophysics Data System (ADS)

Yulmetyev, R. M.; Gafarov, F. M.

1999-12-01

We consider the Markov and non-Markov processes in complex systems by the dynamical information Shannon entropy (DISE) method. The influence and important role of the two mutually dependent channels of entropy alternation (creation or generation of correlation) and anti-correlation (destroying or annihilation of correlation) have been discussed. The developed method has been used for the analysis of the complex systems of various natures: slow neutron scattering in liquid cesium, psychology (short-time numeral and pattern human memory and effect of stress on the dynamical taping-test), random dynamics of RR-intervals in human ECG (problem of diagnosis of various disease of the human cardio-vascular systems), chaotic dynamics of the parameters of financial markets and ecological systems.

Open Quantum Systems and Classical Trajectories

NASA Astrophysics Data System (ADS)

Rebolledo, Rolando

2004-09-01

A Quantum Markov Semigroup consists of a family { T} = ({ T}t)_{t ∈ B R+} of normal ω*- continuous completely positive maps on a von Neumann algebra 𝔐 which preserve the unit and satisfy the semigroup property. This class of semigroups has been extensively used to represent open quantum systems. This article is aimed at studying the existence of a { T} -invariant abelian subalgebra 𝔄 of 𝔐. When this happens, the restriction of { T}t to 𝔄 defines a classical Markov semigroup T = (Tt)t ∈ ∝ + say, associated to a classical Markov process X = (Xt)t ∈ ∝ +. The structure (𝔄, T, X) unravels the quantum Markov semigroup { T} , providing a bridge between open quantum systems and classical stochastic processes.

Modeling haplotype block variation using Markov chains.

PubMed

Greenspan, G; Geiger, D

2006-04-01

Models of background variation in genomic regions form the basis of linkage disequilibrium mapping methods. In this work we analyze a background model that groups SNPs into haplotype blocks and represents the dependencies between blocks by a Markov chain. We develop an error measure to compare the performance of this model against the common model that assumes that blocks are independent. By examining data from the International Haplotype Mapping project, we show how the Markov model over haplotype blocks is most accurate when representing blocks in strong linkage disequilibrium. This contrasts with the independent model, which is rendered less accurate by linkage disequilibrium. We provide a theoretical explanation for this surprising property of the Markov model and relate its behavior to allele diversity.

Markov chains for testing redundant software

NASA Technical Reports Server (NTRS)

White, Allan L.; Sjogren, Jon A.

1988-01-01

A preliminary design for a validation experiment has been developed that addresses several problems unique to assuring the extremely high quality of multiple-version programs in process-control software. The procedure uses Markov chains to model the error states of the multiple version programs. The programs are observed during simulated process-control testing, and estimates are obtained for the transition probabilities between the states of the Markov chain. The experimental Markov chain model is then expanded into a reliability model that takes into account the inertia of the system being controlled. The reliability of the multiple version software is computed from this reliability model at a given confidence level using confidence intervals obtained for the transition probabilities during the experiment. An example demonstrating the method is provided.

Bayesian analysis of non-homogeneous Markov chains: application to mental health data.

PubMed

Sung, Minje; Soyer, Refik; Nhan, Nguyen

2007-07-10

In this paper we present a formal treatment of non-homogeneous Markov chains by introducing a hierarchical Bayesian framework. Our work is motivated by the analysis of correlated categorical data which arise in assessment of psychiatric treatment programs. In our development, we introduce a Markovian structure to describe the non-homogeneity of transition patterns. In doing so, we introduce a logistic regression set-up for Markov chains and incorporate covariates in our model. We present a Bayesian model using Markov chain Monte Carlo methods and develop inference procedures to address issues encountered in the analyses of data from psychiatric treatment programs. Our model and inference procedures are implemented to some real data from a psychiatric treatment study. Copyright 2006 John Wiley & Sons, Ltd.

Counting of oligomers in sequences generated by markov chains for DNA motif discovery.

PubMed

Shan, Gao; Zheng, Wei-Mou

2009-02-01

By means of the technique of the imbedded Markov chain, an efficient algorithm is proposed to exactly calculate first, second moments of word counts and the probability for a word to occur at least once in random texts generated by a Markov chain. A generating function is introduced directly from the imbedded Markov chain to derive asymptotic approximations for the problem. Two Z-scores, one based on the number of sequences with hits and the other on the total number of word hits in a set of sequences, are examined for discovery of motifs on a set of promoter sequences extracted from A. thaliana genome. Source code is available at http://www.itp.ac.cn/zheng/oligo.c.

Modeling Haplotype Block Variation Using Markov Chains

PubMed Central

Greenspan, G.; Geiger, D.

2006-01-01

Models of background variation in genomic regions form the basis of linkage disequilibrium mapping methods. In this work we analyze a background model that groups SNPs into haplotype blocks and represents the dependencies between blocks by a Markov chain. We develop an error measure to compare the performance of this model against the common model that assumes that blocks are independent. By examining data from the International Haplotype Mapping project, we show how the Markov model over haplotype blocks is most accurate when representing blocks in strong linkage disequilibrium. This contrasts with the independent model, which is rendered less accurate by linkage disequilibrium. We provide a theoretical explanation for this surprising property of the Markov model and relate its behavior to allele diversity. PMID:16361244

Validation of the SURE Program, phase 1

NASA Technical Reports Server (NTRS)

Dotson, Kelly J.

1987-01-01

Presented are the results of the first phase in the validation of the SURE (Semi-Markov Unreliability Range Evaluator) program. The SURE program gives lower and upper bounds on the death-state probabilities of a semi-Markov model. With these bounds, the reliability of a semi-Markov model of a fault-tolerant computer system can be analyzed. For the first phase in the validation, fifteen semi-Markov models were solved analytically for the exact death-state probabilities and these solutions compared to the corresponding bounds given by SURE. In every case, the SURE bounds covered the exact solution. The bounds, however, had a tendency to separate in cases where the recovery rate was slow or the fault arrival rate was fast.

Influence of credit scoring on the dynamics of Markov chain

NASA Astrophysics Data System (ADS)

Galina, Timofeeva

2015-11-01

Markov processes are widely used to model the dynamics of a credit portfolio and forecast the portfolio risk and profitability. In the Markov chain model the loan portfolio is divided into several groups with different quality, which determined by presence of indebtedness and its terms. It is proposed that dynamics of portfolio shares is described by a multistage controlled system. The article outlines mathematical formalization of controls which reflect the actions of the bank's management in order to improve the loan portfolio quality. The most important control is the organization of approval procedure of loan applications. The credit scoring is studied as a control affecting to the dynamic system. Different formalizations of "good" and "bad" consumers are proposed in connection with the Markov chain model.

Markov Modeling of Component Fault Growth over a Derived Domain of Feasible Output Control Effort Modifications

NASA Technical Reports Server (NTRS)

Bole, Brian; Goebel, Kai; Vachtsevanos, George

2012-01-01

This paper introduces a novel Markov process formulation of stochastic fault growth modeling, in order to facilitate the development and analysis of prognostics-based control adaptation. A metric representing the relative deviation between the nominal output of a system and the net output that is actually enacted by an implemented prognostics-based control routine, will be used to define the action space of the formulated Markov process. The state space of the Markov process will be defined in terms of an abstracted metric representing the relative health remaining in each of the system s components. The proposed formulation of component fault dynamics will conveniently relate feasible system output performance modifications to predictions of future component health deterioration.

A Hybrid Seismic Inversion Method for V P/V S Ratio and Its Application to Gas Identification

NASA Astrophysics Data System (ADS)

Guo, Qiang; Zhang, Hongbing; Han, Feilong; Xiao, Wei; Shang, Zuoping

2018-03-01

The ratio of compressional wave velocity to shear wave velocity (V P/V S ratio) has established itself as one of the most important parameters in identifying gas reservoirs. However, considering that seismic inversion process is highly non-linear and geological conditions encountered may be complex, a direct estimation of V P/V S ratio from pre-stack seismic data remains a challenging task. In this paper, we propose a hybrid seismic inversion method to estimate V P/V S ratio directly. In this method, post- and pre-stack inversions are combined in which the pre-stack inversion for V P/V S ratio is driven by the post-stack inversion results (i.e., V P and density). In particular, the V P/V S ratio is considered as a model parameter and is directly inverted from the pre-stack inversion based on the exact Zoeppritz equation. Moreover, anisotropic Markov random field is employed in order to regularise the inversion process as well as taking care of geological structures (boundaries) information. Aided by the proposed hybrid inversion strategy, the directional weighting coefficients incorporated in the anisotropic Markov random field neighbourhoods are quantitatively calculated by the anisotropic diffusion method. The synthetic test demonstrates the effectiveness of the proposed inversion method. In particular, given low quality of the pre-stack data and high heterogeneity of the target layers in the field data, the proposed inversion method reveals the detailed model of V P/V S ratio that can successfully identify the gas-bearing zones.

A method for using real world data in breast cancer modeling.

PubMed

Pobiruchin, Monika; Bochum, Sylvia; Martens, Uwe M; Kieser, Meinhard; Schramm, Wendelin

2016-04-01

Today, hospitals and other health care-related institutions are accumulating a growing bulk of real world clinical data. Such data offer new possibilities for the generation of disease models for the health economic evaluation. In this article, we propose a new approach to leverage cancer registry data for the development of Markov models. Records of breast cancer patients from a clinical cancer registry were used to construct a real world data driven disease model. We describe a model generation process which maps database structures to disease state definitions based on medical expert knowledge. Software was programmed in Java to automatically derive a model structure and transition probabilities. We illustrate our method with the reconstruction of a published breast cancer reference model derived primarily from clinical study data. In doing so, we exported longitudinal patient data from a clinical cancer registry covering eight years. The patient cohort (n=892) comprised HER2-positive and HER2-negative women treated with or without Trastuzumab. The models generated with this method for the respective patient cohorts were comparable to the reference model in their structure and treatment effects. However, our computed disease models reflect a more detailed picture of the transition probabilities, especially for disease free survival and recurrence. Our work presents an approach to extract Markov models semi-automatically using real world data from a clinical cancer registry. Health care decision makers may benefit from more realistic disease models to improve health care-related planning and actions based on their own data. Copyright © 2016 The Authors. Published by Elsevier Inc. All rights reserved.

Audiologist-driven versus patient-driven fine tuning of hearing instruments.

PubMed

Boymans, Monique; Dreschler, Wouter A

2012-03-01

Two methods of fine tuning the initial settings of hearing aids were compared: An audiologist-driven approach--using real ear measurements and a patient-driven fine-tuning approach--using feedback from real-life situations. The patient-driven fine tuning was conducted by employing the Amplifit(®) II system using audiovideo clips. The audiologist-driven fine tuning was based on the NAL-NL1 prescription rule. Both settings were compared using the same hearing aids in two 6-week trial periods following a randomized blinded cross-over design. After each trial period, the settings were evaluated by insertion-gain measurements. Performance was evaluated by speech tests in quiet, in noise, and in time-reversed speech, presented at 0° and with spatially separated sound sources. Subjective results were evaluated using extensive questionnaires and audiovisual video clips. A total of 73 participants were included. On average, higher gain values were found for the audiologist-driven settings than for the patient-driven settings, especially at 1000 and 2000 Hz. Better objective performance was obtained for the audiologist-driven settings for speech perception in quiet and in time-reversed speech. This was supported by better scores on a number of subjective judgments and in the subjective ratings of video clips. The perception of loud sounds scored higher than when patient-driven, but the overall preference was in favor of the audiologist-driven settings for 67% of the participants.

Chaos control and synchronization in Bragg acousto-optic bistable systems driven by a separate chaotic system.

PubMed

Wang, Rong; Gao, Jin-Yue

2005-09-01

In this paper we propose a new scheme to achieve chaos control and synchronization in Bragg acousto-optic bistable systems. In the scheme, we use the output of one system to drive two identical chaotic systems. Using the maximal conditional Lyapunov exponent (MCLE) as the criterion, we analyze the conditions for realizing chaos synchronization. Numerical calculation shows that the two identical systems in chaos with negative MCLEs and driven by a chaotic system can go into chaotic synchronization whether or not they were in chaos initially. The two systems can go into different periodic states from chaos following an inverse period-doubling bifurcation route as well when driven by a periodic system.

Nonequilibrium thermodynamics of restricted Boltzmann machines.

PubMed

Salazar, Domingos S P

2017-08-01

In this work, we analyze the nonequilibrium thermodynamics of a class of neural networks known as restricted Boltzmann machines (RBMs) in the context of unsupervised learning. We show how the network is described as a discrete Markov process and how the detailed balance condition and the Maxwell-Boltzmann equilibrium distribution are sufficient conditions for a complete thermodynamics description, including nonequilibrium fluctuation theorems. Numerical simulations in a fully trained RBM are performed and the heat exchange fluctuation theorem is verified with excellent agreement to the theory. We observe how the contrastive divergence functional, mostly used in unsupervised learning of RBMs, is closely related to nonequilibrium thermodynamic quantities. We also use the framework to interpret the estimation of the partition function of RBMs with the annealed importance sampling method from a thermodynamics standpoint. Finally, we argue that unsupervised learning of RBMs is equivalent to a work protocol in a system driven by the laws of thermodynamics in the absence of labeled data.

Expected Power-Utility Maximization Under Incomplete Information and with Cox-Process Observations

DOE Office of Scientific and Technical Information (OSTI.GOV)

Fujimoto, Kazufumi, E-mail: m_fuji@kvj.biglobe.ne.jp; Nagai, Hideo, E-mail: nagai@sigmath.es.osaka-u.ac.jp; Runggaldier, Wolfgang J., E-mail: runggal@math.unipd.it

2013-02-15

We consider the problem of maximization of expected terminal power utility (risk sensitive criterion). The underlying market model is a regime-switching diffusion model where the regime is determined by an unobservable factor process forming a finite state Markov process. The main novelty is due to the fact that prices are observed and the portfolio is rebalanced only at random times corresponding to a Cox process where the intensity is driven by the unobserved Markovian factor process as well. This leads to a more realistic modeling for many practical situations, like in markets with liquidity restrictions; on the other hand itmore » considerably complicates the problem to the point that traditional methodologies cannot be directly applied. The approach presented here is specific to the power-utility. For log-utilities a different approach is presented in Fujimoto et al. (Preprint, 2012).« less

Emotion and decision-making: affect-driven belief systems in anxiety and depression.

PubMed

Paulus, Martin P; Yu, Angela J

2012-09-01

Emotion processing and decision-making are integral aspects of daily life. However, our understanding of the interaction between these constructs is limited. In this review, we summarize theoretical approaches that link emotion and decision-making, and focus on research with anxious or depressed individuals to show how emotions can interfere with decision-making. We integrate the emotional framework based on valence and arousal with a Bayesian approach to decision-making in terms of probability and value processing. We discuss how studies of individuals with emotional dysfunctions provide evidence that alterations of decision-making can be viewed in terms of altered probability and value computation. We argue that the probabilistic representation of belief states in the context of partially observable Markov decision processes provides a useful approach to examine alterations in probability and value representation in individuals with anxiety and depression, and outline the broader implications of this approach. Copyright © 2012. Published by Elsevier Ltd.

Emotion and decision-making: affect-driven belief systems in anxiety and depression

PubMed Central

Paulus, Martin P.; Yu, Angela J.

2012-01-01

Emotion processing and decision-making are integral aspects of daily life. However, our understanding of the interaction between these constructs is limited. In this review, we summarize theoretical approaches to the link between emotion and decision-making, and focus on research with anxious or depressed individuals that reveals how emotions can interfere with decision-making. We integrate the emotional framework based on valence and arousal with a Bayesian approach to decision-making in terms of probability and value processing. We then discuss how studies of individuals with emotional dysfunctions provide evidence that alterations of decision-making can be viewed in terms of altered probability and value computation. We argue that the probabilistic representation of belief states in the context of partially observable Markov decision processes provides a useful approach to examine alterations in probability and value representation in individuals with anxiety and depression and outline the broader implications of this approach. PMID:22898207

Markov modeling for the neurosurgeon: a review of the literature and an introduction to cost-effectiveness research.

PubMed

Wali, Arvin R; Brandel, Michael G; Santiago-Dieppa, David R; Rennert, Robert C; Steinberg, Jeffrey A; Hirshman, Brian R; Murphy, James D; Khalessi, Alexander A

2018-05-01

OBJECTIVE Markov modeling is a clinical research technique that allows competing medical strategies to be mathematically assessed in order to identify the optimal allocation of health care resources. The authors present a review of the recently published neurosurgical literature that employs Markov modeling and provide a conceptual framework with which to evaluate, critique, and apply the findings generated from health economics research. METHODS The PubMed online database was searched to identify neurosurgical literature published from January 2010 to December 2017 that had utilized Markov modeling for neurosurgical cost-effectiveness studies. Included articles were then assessed with regard to year of publication, subspecialty of neurosurgery, decision analytical techniques utilized, and source information for model inputs. RESULTS A total of 55 articles utilizing Markov models were identified across a broad range of neurosurgical subspecialties. Sixty-five percent of the papers were published within the past 3 years alone. The majority of models derived health transition probabilities, health utilities, and cost information from previously published studies or publicly available information. Only 62% of the studies incorporated indirect costs. Ninety-three percent of the studies performed a 1-way or 2-way sensitivity analysis, and 67% performed a probabilistic sensitivity analysis. A review of the conceptual framework of Markov modeling and an explanation of the different terminology and methodology are provided. CONCLUSIONS As neurosurgeons continue to innovate and identify novel treatment strategies for patients, Markov modeling will allow for better characterization of the impact of these interventions on a patient and societal level. The aim of this work is to equip the neurosurgical readership with the tools to better understand, critique, and apply findings produced from cost-effectiveness research.

A stochastic model for tumor geometry evolution during radiation therapy in cervical cancer

DOE Office of Scientific and Technical Information (OSTI.GOV)

Liu, Yifang; Lee, Chi-Guhn; Chan, Timothy C. Y., E-mail: tcychan@mie.utoronto.ca

2014-02-15

Purpose: To develop mathematical models to predict the evolution of tumor geometry in cervical cancer undergoing radiation therapy. Methods: The authors develop two mathematical models to estimate tumor geometry change: a Markov model and an isomorphic shrinkage model. The Markov model describes tumor evolution by investigating the change in state (either tumor or nontumor) of voxels on the tumor surface. It assumes that the evolution follows a Markov process. Transition probabilities are obtained using maximum likelihood estimation and depend on the states of neighboring voxels. The isomorphic shrinkage model describes tumor shrinkage or growth in terms of layers of voxelsmore » on the tumor surface, instead of modeling individual voxels. The two proposed models were applied to data from 29 cervical cancer patients treated at Princess Margaret Cancer Centre and then compared to a constant volume approach. Model performance was measured using sensitivity and specificity. Results: The Markov model outperformed both the isomorphic shrinkage and constant volume models in terms of the trade-off between sensitivity (target coverage) and specificity (normal tissue sparing). Generally, the Markov model achieved a few percentage points in improvement in either sensitivity or specificity compared to the other models. The isomorphic shrinkage model was comparable to the Markov approach under certain parameter settings. Convex tumor shapes were easier to predict. Conclusions: By modeling tumor geometry change at the voxel level using a probabilistic model, improvements in target coverage and normal tissue sparing are possible. Our Markov model is flexible and has tunable parameters to adjust model performance to meet a range of criteria. Such a model may support the development of an adaptive paradigm for radiation therapy of cervical cancer.« less

Revisiting Temporal Markov Chains for Continuum modeling of Transport in Porous Media

NASA Astrophysics Data System (ADS)

Delgoshaie, A. H.; Jenny, P.; Tchelepi, H.

2017-12-01

The transport of fluids in porous media is dominated by flow­-field heterogeneity resulting from the underlying permeability field. Due to the high uncertainty in the permeability field, many realizations of the reference geological model are used to describe the statistics of the transport phenomena in a Monte Carlo (MC) framework. There has been strong interest in working with stochastic formulations of the transport that are different from the standard MC approach. Several stochastic models based on a velocity process for tracer particle trajectories have been proposed. Previous studies have shown that for high variances of the log-conductivity, the stochastic models need to account for correlations between consecutive velocity transitions to predict dispersion accurately. The correlated velocity models proposed in the literature can be divided into two general classes of temporal and spatial Markov models. Temporal Markov models have been applied successfully to tracer transport in both the longitudinal and transverse directions. These temporal models are Stochastic Differential Equations (SDEs) with very specific drift and diffusion terms tailored for a specific permeability correlation structure. The drift and diffusion functions devised for a certain setup would not necessarily be suitable for a different scenario, (e.g., a different permeability correlation structure). The spatial Markov models are simple discrete Markov chains that do not require case specific assumptions. However, transverse spreading of contaminant plumes has not been successfully modeled with the available correlated spatial models. Here, we propose a temporal discrete Markov chain to model both the longitudinal and transverse dispersion in a two-dimensional domain. We demonstrate that these temporal Markov models are valid for different correlation structures without modification. Similar to the temporal SDEs, the proposed model respects the limited asymptotic transverse spreading of the plume in two-dimensional problems.

Markovian prediction of future values for food grains in the economic survey

NASA Astrophysics Data System (ADS)

Sathish, S.; Khadar Babu, S. K.

2017-11-01

Now-a-days prediction and forecasting are plays a vital role in research. For prediction, regression is useful to predict the future value and current value on production process. In this paper, we assume food grain production exhibit Markov chain dependency and time homogeneity. The economic generative performance evaluation the balance time artificial fertilization different level in Estrusdetection using a daily Markov chain model. Finally, Markov process prediction gives better performance compare with Regression model.

Computing rates of Markov models of voltage-gated ion channels by inverting partial differential equations governing the probability density functions of the conducting and non-conducting states.

PubMed

Tveito, Aslak; Lines, Glenn T; Edwards, Andrew G; McCulloch, Andrew

2016-07-01

Markov models are ubiquitously used to represent the function of single ion channels. However, solving the inverse problem to construct a Markov model of single channel dynamics from bilayer or patch-clamp recordings remains challenging, particularly for channels involving complex gating processes. Methods for solving the inverse problem are generally based on data from voltage clamp measurements. Here, we describe an alternative approach to this problem based on measurements of voltage traces. The voltage traces define probability density functions of the functional states of an ion channel. These probability density functions can also be computed by solving a deterministic system of partial differential equations. The inversion is based on tuning the rates of the Markov models used in the deterministic system of partial differential equations such that the solution mimics the properties of the probability density function gathered from (pseudo) experimental data as well as possible. The optimization is done by defining a cost function to measure the difference between the deterministic solution and the solution based on experimental data. By evoking the properties of this function, it is possible to infer whether the rates of the Markov model are identifiable by our method. We present applications to Markov model well-known from the literature. Copyright © 2016 The Authors. Published by Elsevier Inc. All rights reserved.

Vlasov dynamics of periodically driven systems

NASA Astrophysics Data System (ADS)

Banerjee, Soumyadip; Shah, Kushal

2018-04-01

Analytical solutions of the Vlasov equation for periodically driven systems are of importance in several areas of plasma physics and dynamical systems and are usually approximated using ponderomotive theory. In this paper, we derive the plasma distribution function predicted by ponderomotive theory using Hamiltonian averaging theory and compare it with solutions obtained by the method of characteristics. Our results show that though ponderomotive theory is relatively much easier to use, its predictions are very restrictive and are likely to be very different from the actual distribution function of the system. We also analyse all possible initial conditions which lead to periodic solutions of the Vlasov equation for periodically driven systems and conjecture that the irreducible polynomial corresponding to the initial condition must only have squares of the spatial and momentum coordinate. The resulting distribution function for other initial conditions is aperiodic and can lead to complex relaxation processes within the plasma.

A statistical model investigating the prevalence of tuberculosis in New York City using counting processes with two change-points

PubMed Central

ACHCAR, J. A.; MARTINEZ, E. Z.; RUFFINO-NETTO, A.; PAULINO, C. D.; SOARES, P.

2008-01-01

SUMMARY We considered a Bayesian analysis for the prevalence of tuberculosis cases in New York City from 1970 to 2000. This counting dataset presented two change-points during this period. We modelled this counting dataset considering non-homogeneous Poisson processes in the presence of the two-change points. A Bayesian analysis for the data is considered using Markov chain Monte Carlo methods. Simulated Gibbs samples for the parameters of interest were obtained using WinBugs software. PMID:18346287

Improving Self-Assembly by Varying the Temperature Periodically with Time

NASA Astrophysics Data System (ADS)

Raz, Oren; Jarzynski, Christopher

Self-assembly (SA) is the process by which basic components organize into a larger structure without external guidance. These processes are common in Nature, and also have technological applications, e.g. growing a crystal with a specific structure. So far, artificial SA processes have been designed mostly using diffusive building blocks with high specificity and directionality. The formation of the self-assembled structures is then driven by free-energy minimization into a thermodynamically stable state. In an alternative approach to SA, macroscopic parameters such as temperature, pressure, pH, magnetic field etc., are varied periodically with time. In this case, the SA structures are the stable periodic states of the driven system. Currently there are no design principles for periodically driven SA, other than in the limits of fast or weak driving. We present guiding ideas for self-assembly under periodic driving. As an example, we show a particular case in which self-assembly errors can be dramatically reduced by varying a system's temperature periodically with time. James S. McDonnell Foundation, and the US National Science Foundation: DMR-1506969.

Multiensemble Markov models of molecular thermodynamics and kinetics.

PubMed

Wu, Hao; Paul, Fabian; Wehmeyer, Christoph; Noé, Frank

2016-06-07

We introduce the general transition-based reweighting analysis method (TRAM), a statistically optimal approach to integrate both unbiased and biased molecular dynamics simulations, such as umbrella sampling or replica exchange. TRAM estimates a multiensemble Markov model (MEMM) with full thermodynamic and kinetic information at all ensembles. The approach combines the benefits of Markov state models-clustering of high-dimensional spaces and modeling of complex many-state systems-with those of the multistate Bennett acceptance ratio of exploiting biased or high-temperature ensembles to accelerate rare-event sampling. TRAM does not depend on any rate model in addition to the widely used Markov state model approximation, but uses only fundamental relations such as detailed balance and binless reweighting of configurations between ensembles. Previous methods, including the multistate Bennett acceptance ratio, discrete TRAM, and Markov state models are special cases and can be derived from the TRAM equations. TRAM is demonstrated by efficiently computing MEMMs in cases where other estimators break down, including the full thermodynamics and rare-event kinetics from high-dimensional simulation data of an all-atom protein-ligand binding model.

Multiensemble Markov models of molecular thermodynamics and kinetics

PubMed Central

Wu, Hao; Paul, Fabian; Noé, Frank

2016-01-01

We introduce the general transition-based reweighting analysis method (TRAM), a statistically optimal approach to integrate both unbiased and biased molecular dynamics simulations, such as umbrella sampling or replica exchange. TRAM estimates a multiensemble Markov model (MEMM) with full thermodynamic and kinetic information at all ensembles. The approach combines the benefits of Markov state models—clustering of high-dimensional spaces and modeling of complex many-state systems—with those of the multistate Bennett acceptance ratio of exploiting biased or high-temperature ensembles to accelerate rare-event sampling. TRAM does not depend on any rate model in addition to the widely used Markov state model approximation, but uses only fundamental relations such as detailed balance and binless reweighting of configurations between ensembles. Previous methods, including the multistate Bennett acceptance ratio, discrete TRAM, and Markov state models are special cases and can be derived from the TRAM equations. TRAM is demonstrated by efficiently computing MEMMs in cases where other estimators break down, including the full thermodynamics and rare-event kinetics from high-dimensional simulation data of an all-atom protein–ligand binding model. PMID:27226302

Fuzzy Markov random fields versus chains for multispectral image segmentation.

PubMed

Salzenstein, Fabien; Collet, Christophe

2006-11-01

This paper deals with a comparison of recent statistical models based on fuzzy Markov random fields and chains for multispectral image segmentation. The fuzzy scheme takes into account discrete and continuous classes which model the imprecision of the hidden data. In this framework, we assume the dependence between bands and we express the general model for the covariance matrix. A fuzzy Markov chain model is developed in an unsupervised way. This method is compared with the fuzzy Markovian field model previously proposed by one of the authors. The segmentation task is processed with Bayesian tools, such as the well-known MPM (Mode of Posterior Marginals) criterion. Our goal is to compare the robustness and rapidity for both methods (fuzzy Markov fields versus fuzzy Markov chains). Indeed, such fuzzy-based procedures seem to be a good answer, e.g., for astronomical observations when the patterns present diffuse structures. Moreover, these approaches allow us to process missing data in one or several spectral bands which correspond to specific situations in astronomy. To validate both models, we perform and compare the segmentation on synthetic images and raw multispectral astronomical data.

On buffer overflow duration in a finite-capacity queueing system with multiple vacation policy

NASA Astrophysics Data System (ADS)

Kempa, Wojciech M.

2017-12-01

A finite-buffer queueing system with Poisson arrivals and generally distributed processing times, operating under multiple vacation policy, is considered. Each time when the system becomes empty, the service station takes successive independent and identically distributed vacation periods, until, at the completion epoch of one of them, at least one job waiting for service is detected in the buffer. Applying analytical approach based on the idea of embedded Markov chain, integral equations and linear algebra, the compact-form representation for the cumulative distribution function (CDF for short) of the first buffer overflow duration is found. Hence, the formula for the CDF of next such periods is obtained. Moreover, probability distributions of the number of job losses in successive buffer overflow periods are found. The considered queueing system can be efficienly applied in modelling energy saving mechanisms in wireless network communication.

A Markov Chain Model for Changes in Users' Assessment of Search Results.

PubMed

Zhitomirsky-Geffet, Maayan; Bar-Ilan, Judit; Levene, Mark

2016-01-01

Previous research shows that users tend to change their assessment of search results over time. This is a first study that investigates the factors and reasons for these changes, and describes a stochastic model of user behaviour that may explain these changes. In particular, we hypothesise that most of the changes are local, i.e. between results with similar or close relevance to the query, and thus belong to the same"coarse" relevance category. According to the theory of coarse beliefs and categorical thinking, humans tend to divide the range of values under consideration into coarse categories, and are thus able to distinguish only between cross-category values but not within them. To test this hypothesis we conducted five experiments with about 120 subjects divided into 3 groups. Each student in every group was asked to rank and assign relevance scores to the same set of search results over two or three rounds, with a period of three to nine weeks between each round. The subjects of the last three-round experiment were then exposed to the differences in their judgements and were asked to explain them. We make use of a Markov chain model to measure change in users' judgments between the different rounds. The Markov chain demonstrates that the changes converge, and that a majority of the changes are local to a neighbouring relevance category. We found that most of the subjects were satisfied with their changes, and did not perceive them as mistakes but rather as a legitimate phenomenon, since they believe that time has influenced their relevance assessment. Both our quantitative analysis and user comments support the hypothesis of the existence of coarse relevance categories resulting from categorical thinking in the context of user evaluation of search results.

A Markov Chain Model for Changes in Users’ Assessment of Search Results

PubMed Central

Zhitomirsky-Geffet, Maayan; Bar-Ilan, Judit; Levene, Mark

2016-01-01

Previous research shows that users tend to change their assessment of search results over time. This is a first study that investigates the factors and reasons for these changes, and describes a stochastic model of user behaviour that may explain these changes. In particular, we hypothesise that most of the changes are local, i.e. between results with similar or close relevance to the query, and thus belong to the same”coarse” relevance category. According to the theory of coarse beliefs and categorical thinking, humans tend to divide the range of values under consideration into coarse categories, and are thus able to distinguish only between cross-category values but not within them. To test this hypothesis we conducted five experiments with about 120 subjects divided into 3 groups. Each student in every group was asked to rank and assign relevance scores to the same set of search results over two or three rounds, with a period of three to nine weeks between each round. The subjects of the last three-round experiment were then exposed to the differences in their judgements and were asked to explain them. We make use of a Markov chain model to measure change in users’ judgments between the different rounds. The Markov chain demonstrates that the changes converge, and that a majority of the changes are local to a neighbouring relevance category. We found that most of the subjects were satisfied with their changes, and did not perceive them as mistakes but rather as a legitimate phenomenon, since they believe that time has influenced their relevance assessment. Both our quantitative analysis and user comments support the hypothesis of the existence of coarse relevance categories resulting from categorical thinking in the context of user evaluation of search results. PMID:27171426

Analysis of current-driven oscillatory dynamics of single-layer homoepitaxial islands on crystalline conducting substrates

NASA Astrophysics Data System (ADS)

Dasgupta, Dwaipayan; Kumar, Ashish; Maroudas, Dimitrios

2018-03-01

We report results of a systematic study on the complex oscillatory current-driven dynamics of single-layer homoepitaxial islands on crystalline substrate surfaces and the dependence of this driven dynamical behavior on important physical parameters, including island size, substrate surface orientation, and direction of externally applied electric field. The analysis is based on a nonlinear model of driven island edge morphological evolution that accounts for curvature-driven edge diffusion, edge electromigration, and edge diffusional anisotropy. Using a linear theory of island edge morphological stability, we calculate a critical island size at which the island's equilibrium edge shape becomes unstable, which sets a lower bound for the onset of time-periodic oscillatory dynamical response. Using direct dynamical simulations, we study the edge morphological dynamics of current-driven single-layer islands at larger-than-critical size, and determine the actual island size at which the migrating islands undergo a transition from steady to time-periodic asymptotic states through a subcritical Hopf bifurcation. At the highest symmetry of diffusional anisotropy examined, on {111} surfaces of face-centered cubic crystalline substrates, we find that more complex stable oscillatory states can be reached through period-doubling bifurcation at island sizes larger than those at the Hopf points. We characterize in detail the island morphology and dynamical response at the stable time-periodic asymptotic states, determine the range of stability of these oscillatory states terminated by island breakup, and explain the morphological features of the stable oscillating islands on the basis of linear stability theory.

The Role of Climatic and Density Dependent Factors in Shaping Mosquito Population Dynamics: The Case of Culex pipiens in Northwestern Italy

PubMed Central

Giacobini, Mario; Pugliese, Andrea; Merler, Stefano; Rosà, Roberto

2016-01-01

Culex pipiens mosquito is a species widely spread across Europe and represents a competent vector for many arboviruses such as West Nile virus (WNV), which has been recently circulating in many European countries, causing hundreds of human cases. In order to identify the main determinants of the high heterogeneity in Cx. pipiens abundance observed in Piedmont region (Northwestern Italy) among different seasons, we developed a density-dependent stochastic model that takes explicitly into account the role played by temperature, which affects both developmental and mortality rates of different life stages. The model was calibrated with a Markov chain Monte Carlo approach exploring the likelihood of recorded capture data gathered in the study area from 2000 to 2011; in this way, we disentangled the role played by different seasonal eco-climatic factors in shaping the vector abundance. Illustrative simulations have been performed to forecast likely changes if temperature or density–dependent inputs would change. Our analysis suggests that inter-seasonal differences in the mosquito dynamics are largely driven by different temporal patterns of temperature and seasonal-specific larval carrying capacities. Specifically, high temperatures during early spring hasten the onset of the breeding season and increase population abundance in that period, while, high temperatures during the summer can decrease population size by increasing adult mortality. Higher densities of adult mosquitoes are associated with higher larval carrying capacities, which are positively correlated with spring precipitations. Finally, an increase in larval carrying capacity is expected to proportionally increase adult mosquito abundance. PMID:27105065

Seasonal climate variation and caribou availability: Modeling sequential movement using satellite-relocation data

USGS Publications Warehouse

Nicolson, Craig; Berman, Matthew; West, Colin Thor; Kofinas, Gary P.; Griffith, Brad; Russell, Don; Dugan, Darcy

2013-01-01

Livelihood systems that depend on mobile resources must constantly adapt to change. For people living in permanent settlements, environmental changes that affect the distribution of a migratory species may reduce the availability of a primary food source, with the potential to destabilize the regional social-ecological system. Food security for Arctic indigenous peoples harvesting barren ground caribou (Rangifer tarandus granti) depends on movement patterns of migratory herds. Quantitative assessments of physical, ecological, and social effects on caribou distribution have proven difficult because of the significant interannual variability in seasonal caribou movement patterns. We developed and evaluated a modeling approach for simulating the distribution of a migratory herd throughout its annual cycle over a multiyear period. Beginning with spatial and temporal scales developed in previous studies of the Porcupine Caribou Herd of Canada and Alaska, we used satellite collar locations to compute and analyze season-by-season probabilities of movement of animals between habitat zones under two alternative weather conditions for each season. We then built a set of transition matrices from these movement probabilities, and simulated the sequence of movements across the landscape as a Markov process driven by externally imposed seasonal weather states. Statistical tests showed that the predicted distributions of caribou were consistent with observed distributions, and significantly correlated with subsistence harvest levels for three user communities. Our approach could be applied to other caribou herds and could be adapted for simulating the distribution of other ungulates and species with similarly large interannual variability in the use of their range.

Prediction and generation of binary Markov processes: Can a finite-state fox catch a Markov mouse?

NASA Astrophysics Data System (ADS)

Ruebeck, Joshua B.; James, Ryan G.; Mahoney, John R.; Crutchfield, James P.

2018-01-01

Understanding the generative mechanism of a natural system is a vital component of the scientific method. Here, we investigate one of the fundamental steps toward this goal by presenting the minimal generator of an arbitrary binary Markov process. This is a class of processes whose predictive model is well known. Surprisingly, the generative model requires three distinct topologies for different regions of parameter space. We show that a previously proposed generator for a particular set of binary Markov processes is, in fact, not minimal. Our results shed the first quantitative light on the relative (minimal) costs of prediction and generation. We find, for instance, that the difference between prediction and generation is maximized when the process is approximately independently, identically distributed.

Cascade heterogeneous face sketch-photo synthesis via dual-scale Markov Network

NASA Astrophysics Data System (ADS)

Yao, Saisai; Chen, Zhenxue; Jia, Yunyi; Liu, Chengyun

2018-03-01

Heterogeneous face sketch-photo synthesis is an important and challenging task in computer vision, which has widely applied in law enforcement and digital entertainment. According to the different synthesis results based on different scales, this paper proposes a cascade sketch-photo synthesis method via dual-scale Markov Network. Firstly, Markov Network with larger scale is used to synthesise the initial sketches and the local vertical and horizontal neighbour search (LVHNS) method is used to search for the neighbour patches of test patches in training set. Then, the initial sketches and test photos are jointly entered into smaller scale Markov Network. Finally, the fine sketches are obtained after cascade synthesis process. Extensive experimental results on various databases demonstrate the superiority of the proposed method compared with several state-of-the-art methods.

Wave chaos in the elastic disk.

PubMed

Sondergaard, Niels; Tanner, Gregor

2002-12-01

The relation between the elastic wave equation for plane, isotropic bodies and an underlying classical ray dynamics is investigated. We study, in particular, the eigenfrequencies of an elastic disk with free boundaries and their connection to periodic rays inside the circular domain. Even though the problem is separable, wave mixing between the shear and pressure component of the wave field at the boundary leads to an effective stochastic part in the ray dynamics. This introduces phenomena typically associated with classical chaos as, for example, an exponential increase in the number of periodic orbits. Classically, the problem can be decomposed into an integrable part and a simple binary Markov process. Similarly, the wave equation can, in the high-frequency limit, be mapped onto a quantum graph. Implications of this result for the level statistics are discussed. Furthermore, a periodic trace formula is derived from the scattering matrix based on the inside-outside duality between eigenmodes and scattering solutions and periodic orbits are identified by Fourier transforming the spectral density.

First and second order semi-Markov chains for wind speed modeling

NASA Astrophysics Data System (ADS)

Prattico, F.; Petroni, F.; D'Amico, G.

2012-04-01

The increasing interest in renewable energy leads scientific research to find a better way to recover most of the available energy. Particularly, the maximum energy recoverable from wind is equal to 59.3% of that available (Betz law) at a specific pitch angle and when the ratio between the wind speed in output and in input is equal to 1/3. The pitch angle is the angle formed between the airfoil of the blade of the wind turbine and the wind direction. Old turbine and a lot of that actually marketed, in fact, have always the same invariant geometry of the airfoil. This causes that wind turbines will work with an efficiency that is lower than 59.3%. New generation wind turbines, instead, have a system to variate the pitch angle by rotating the blades. This system able the wind turbines to recover, at different wind speed, always the maximum energy, working in Betz limit at different speed ratios. A powerful system control of the pitch angle allows the wind turbine to recover better the energy in transient regime. A good stochastic model for wind speed is then needed to help both the optimization of turbine design and to assist the system control to predict the value of the wind speed to positioning the blades quickly and correctly. The possibility to have synthetic data of wind speed is a powerful instrument to assist designer to verify the structures of the wind turbines or to estimate the energy recoverable from a specific site. To generate synthetic data, Markov chains of first or higher order are often used [1,2,3]. In particular in [3] is presented a comparison between a first-order Markov chain and a second-order Markov chain. A similar work, but only for the first-order Markov chain, is conduced by [2], presenting the probability transition matrix and comparing the energy spectral density and autocorrelation of real and synthetic wind speed data. A tentative to modeling and to join speed and direction of wind is presented in [1], by using two models, first-order Markov chain with different number of states, and Weibull distribution. All this model use Markov chains to generate synthetic wind speed time series but the search for a better model is still open. Approaching this issue, we applied new models which are generalization of Markov models. More precisely we applied semi-Markov models to generate synthetic wind speed time series. Semi-Markov processes (SMP) are a wide class of stochastic processes which generalize at the same time both Markov chains and renewal processes. Their main advantage is that of using whatever type of waiting time distribution for modeling the time to have a transition from one state to another one. This major flexibility has a price to pay: availability of data to estimate the parameters of the model which are more numerous. Data availability is not an issue in wind speed studies, therefore, semi-Markov models can be used in a statistical efficient way. In this work we present three different semi-Markov chain models: the first one is a first-order SMP where the transition probabilities from two speed states (at time Tn and Tn-1) depend on the initial state (the state at Tn-1), final state (the state at Tn) and on the waiting time (given by t=Tn-Tn-1), the second model is a second order SMP where we consider the transition probabilities as depending also on the state the wind speed was before the initial state (which is the state at Tn-2) and the last one is still a second order SMP where the transition probabilities depends on the three states at Tn-2,Tn-1 and Tn and on the waiting times t_1=Tn-1-Tn-2 and t_2=Tn-Tn-1. The three models are used to generate synthetic time series for wind speed by means of Monte Carlo simulations and the time lagged autocorrelation is used to compare statistical properties of the proposed models with those of real data and also with a time series generated though a simple Markov chain. [1] F. Youcef Ettoumi, H. Sauvageot, A.-E.-H. Adane, Statistical bivariate modeling of wind using first-order Markov chain and Weibull distribution, Renewable Energy, 28/2003 1787-1802. [2] A. Shamshad, M.A. Bawadi, W.M.W. Wan Hussin, T.A. Majid, S.A.M. Sanusi, First and second order Markov chain models for synthetic generation of wind speed time series, Energy 30/2005 693-708. [3] H. Nfaoui, H. Essiarab, A.A.M. Sayigh, A stochastic Markov chain model for simulating wind speed time series at Tangiers, Morocco, Renewable Energy 29/2004, 1407-1418.

Electroviscous effect and electrokinetic energy conversion in time periodic pressure-driven flow through a parallel-plate nanochannel with surface charge-dependent slip

NASA Astrophysics Data System (ADS)

Buren, Mandula; Jian, Yongjun; Zhao, Yingchun; Chang, Long

2018-05-01

In this paper we analytically investigate the electroviscous effect and electrokinetic energy conversion in the time periodic pressure-driven flow of an incompressible viscous Newtonian liquid through a parallel-plate nanochannel with surface charge-dependent slip. Analytical and semi-analytical solutions for electric potential, velocity and streaming electric field are obtained and are utilized to compute electrokinetic energy conversion efficiency. The results show that velocity amplitude and energy conversion efficiency are reduced when the effect of surface charge on slip length is considered. The surface charge effect increases with zeta potential and ionic concentration. In addition, the energy conversion efficiency is large when the ratio of channel half-height to the electric double layer thickness is small. The boundary slip results in a large increase in energy conversion. Higher values of the frequency of pressure pulsation lead to higher values of the energy conversion efficiency. We also obtain the energy conversion efficiency in constant pressure-driven flow and find that the energy conversion efficiency in periodical pressure-driven flow becomes larger than that in constant pressure-driven flow when the frequency is large enough.

Diagonal couplings of quantum Markov chains

NASA Astrophysics Data System (ADS)

Kümmerer, Burkhard; Schwieger, Kay

2016-05-01

In this paper we extend the coupling method from classical probability theory to quantum Markov chains on atomic von Neumann algebras. In particular, we establish a coupling inequality, which allow us to estimate convergence rates by analyzing couplings. For a given tensor dilation we construct a self-coupling of a Markov operator. It turns out that the coupling is a dual version of the extended dual transition operator studied by Gohm et al. We deduce that this coupling is successful if and only if the dilation is asymptotically complete.

Developing a Markov Model for Forecasting End Strength of Selected Marine Corps Reserve (SMCR) Officers

DTIC Science & Technology

2013-03-01

moving average ( ARIMA ) model because the data is not a times series. The best a manpower planner can do at this point is to make an educated assumption...MARKOV MODEL FOR FORECASTING END STRENGTH OF SELECTED MARINE CORPS RESERVE (SMCR) OFFICERS by Anthony D. Licari March 2013 Thesis Advisor...March 2013 3. REPORT TYPE AND DATES COVERED Master’s Thesis 4. TITLE AND SUBTITLE DEVELOPING A MARKOV MODEL FOR FORECASTING END STRENGTH OF

Transition records of stationary Markov chains.

PubMed

Naudts, Jan; Van der Straeten, Erik

2006-10-01

In any Markov chain with finite state space the distribution of transition records always belongs to the exponential family. This observation is used to prove a fluctuation theorem, and to show that the dynamical entropy of a stationary Markov chain is linear in the number of steps. Three applications are discussed. A known result about entropy production is reproduced. A thermodynamic relation is derived for equilibrium systems with Metropolis dynamics. Finally, a link is made with recent results concerning a one-dimensional polymer model.

Maximum Kolmogorov-Sinai Entropy Versus Minimum Mixing Time in Markov Chains

NASA Astrophysics Data System (ADS)

Mihelich, M.; Dubrulle, B.; Paillard, D.; Kral, Q.; Faranda, D.

2018-01-01

We establish a link between the maximization of Kolmogorov Sinai entropy (KSE) and the minimization of the mixing time for general Markov chains. Since the maximisation of KSE is analytical and easier to compute in general than mixing time, this link provides a new faster method to approximate the minimum mixing time dynamics. It could be interesting in computer sciences and statistical physics, for computations that use random walks on graphs that can be represented as Markov chains.

Markov and semi-Markov processes as a failure rate

DOE Office of Scientific and Technical Information (OSTI.GOV)

Grabski, Franciszek

2016-06-08

In this paper the reliability function is defined by the stochastic failure rate process with a non negative and right continuous trajectories. Equations for the conditional reliability functions of an object, under assumption that the failure rate is a semi-Markov process with an at most countable state space are derived. A proper theorem is presented. The linear systems of equations for the appropriate Laplace transforms allow to find the reliability functions for the alternating, the Poisson and the Furry-Yule failure rate processes.

Developing a statistically powerful measure for quartet tree inference using phylogenetic identities and Markov invariants.

PubMed

Sumner, Jeremy G; Taylor, Amelia; Holland, Barbara R; Jarvis, Peter D

2017-12-01

Recently there has been renewed interest in phylogenetic inference methods based on phylogenetic invariants, alongside the related Markov invariants. Broadly speaking, both these approaches give rise to polynomial functions of sequence site patterns that, in expectation value, either vanish for particular evolutionary trees (in the case of phylogenetic invariants) or have well understood transformation properties (in the case of Markov invariants). While both approaches have been valued for their intrinsic mathematical interest, it is not clear how they relate to each other, and to what extent they can be used as practical tools for inference of phylogenetic trees. In this paper, by focusing on the special case of binary sequence data and quartets of taxa, we are able to view these two different polynomial-based approaches within a common framework. To motivate the discussion, we present three desirable statistical properties that we argue any invariant-based phylogenetic method should satisfy: (1) sensible behaviour under reordering of input sequences; (2) stability as the taxa evolve independently according to a Markov process; and (3) explicit dependence on the assumption of a continuous-time process. Motivated by these statistical properties, we develop and explore several new phylogenetic inference methods. In particular, we develop a statistically bias-corrected version of the Markov invariants approach which satisfies all three properties. We also extend previous work by showing that the phylogenetic invariants can be implemented in such a way as to satisfy property (3). A simulation study shows that, in comparison to other methods, our new proposed approach based on bias-corrected Markov invariants is extremely powerful for phylogenetic inference. The binary case is of particular theoretical interest as-in this case only-the Markov invariants can be expressed as linear combinations of the phylogenetic invariants. A wider implication of this is that, for models with more than two states-for example DNA sequence alignments with four-state models-we find that methods which rely on phylogenetic invariants are incapable of satisfying all three of the stated statistical properties. This is because in these cases the relevant Markov invariants belong to a class of polynomials independent from the phylogenetic invariants.

Longitudinal aerodynamic characteristics of light, twin-engine, propeller-driven airplanes

NASA Technical Reports Server (NTRS)

Wolowicz, C. H.; Yancey, R. B.

1972-01-01

Representative state-of-the-art analytical procedures and design data for predicting the longitudinal static and dynamic stability and control characteristics of light, propeller-driven airplanes are presented. Procedures for predicting drag characteristics are also included. The procedures are applied to a twin-engine, propeller-driven airplane in the clean configuration from zero lift to stall conditions. The calculated characteristics are compared with wind-tunnel and flight data. Included in the comparisons are level-flight trim characteristics, period and damping of the short-period oscillatory mode, and windup-turn characteristics. All calculations are documented.

A simplified parsimonious higher order multivariate Markov chain model

NASA Astrophysics Data System (ADS)

Wang, Chao; Yang, Chuan-sheng

2017-09-01

In this paper, a simplified parsimonious higher-order multivariate Markov chain model (SPHOMMCM) is presented. Moreover, parameter estimation method of TPHOMMCM is give. Numerical experiments shows the effectiveness of TPHOMMCM.

Study of environmental sound source identification based on hidden Markov model for robust speech recognition

NASA Astrophysics Data System (ADS)

Nishiura, Takanobu; Nakamura, Satoshi

2003-10-01

Humans communicate with each other through speech by focusing on the target speech among environmental sounds in real acoustic environments. We can easily identify the target sound from other environmental sounds. For hands-free speech recognition, the identification of the target speech from environmental sounds is imperative. This mechanism may also be important for a self-moving robot to sense the acoustic environments and communicate with humans. Therefore, this paper first proposes hidden Markov model (HMM)-based environmental sound source identification. Environmental sounds are modeled by three states of HMMs and evaluated using 92 kinds of environmental sounds. The identification accuracy was 95.4%. This paper also proposes a new HMM composition method that composes speech HMMs and an HMM of categorized environmental sounds for robust environmental sound-added speech recognition. As a result of the evaluation experiments, we confirmed that the proposed HMM composition outperforms the conventional HMM composition with speech HMMs and a noise (environmental sound) HMM trained using noise periods prior to the target speech in a captured signal. [Work supported by Ministry of Public Management, Home Affairs, Posts and Telecommunications of Japan.

Cosmic Ray Propagation through the Magnetic Fields of the Galaxy with Extended Halo

NASA Technical Reports Server (NTRS)

Zhang, Ming

2005-01-01

In this project we perform theoretical studies of 3-dimensional cosmic ray propagation in magnetic field configurations of the Galaxy with an extended halo. We employ our newly developed Markov stochastic process methods to solve the diffusive cosmic ray transport equation. We seek to understand observations of cosmic ray spectra, composition under the constraints of the observations of diffuse gamma ray and radio emission from the Galaxy. The model parameters are directly are related to properties of our Galaxy, such as the size of the Galactic halo, particle transport in Galactic magnetic fields, distribution of interstellar gas, primary cosmic ray source distribution and their confinement in the Galaxy. The core of this investigation is the development of software for cosmic ray propagation models with the Markov stochastic process approach. Values of important model parameters for the halo diffusion model are examined in comparison with observations of cosmic ray spectra, composition and the diffuse gamma-ray background. This report summarizes our achievement in the grant period at the Florida Institute of Technology. Work at the co-investigator's institution, the University of New Hampshire, under a companion grant, will be covered in detail by a separate report.

Probabilistic reasoning over seismic RMS time series: volcano monitoring through HMMs and SAX technique

NASA Astrophysics Data System (ADS)

Aliotta, M. A.; Cassisi, C.; Prestifilippo, M.; Cannata, A.; Montalto, P.; Patanè, D.

2014-12-01

During the last years, volcanic activity at Mt. Etna was often characterized by cyclic occurrences of fountains. In the period between January 2011 and June 2013, 38 episodes of lava fountains has been observed. Automatic recognition of the volcano's states related to lava fountain episodes (Quiet, Pre-Fountaining, Fountaining, Post-Fountaining) is very useful for monitoring purposes. We discovered that such states are strongly related to the trend of RMS (Root Mean Square) of the seismic signal recorded in the summit area. In the framework of the project PON SIGMA (Integrated Cloud-Sensor System for Advanced Multirisk Management) work, we tried to model the system generating its sampled values (assuming to be a Markov process and assuming that RMS time series is a stochastic process), by using Hidden Markov models (HMMs), that are a powerful tool for modeling any time-varying series. HMMs analysis seeks to discover the sequence of hidden states from the observed emissions. In our framework, observed emissions are characters generated by SAX (Symbolic Aggregate approXimation) technique. SAX is able to map RMS time series values with discrete literal emissions. Our experiments showed how to predict volcano states by means of SAX and HMMs.

A Stochastic Framework for Evaluating Seizure Prediction Algorithms Using Hidden Markov Models

PubMed Central

Wong, Stephen; Gardner, Andrew B.; Krieger, Abba M.; Litt, Brian

2007-01-01

Responsive, implantable stimulation devices to treat epilepsy are now in clinical trials. New evidence suggests that these devices may be more effective when they deliver therapy before seizure onset. Despite years of effort, prospective seizure prediction, which could improve device performance, remains elusive. In large part, this is explained by lack of agreement on a statistical framework for modeling seizure generation and a method for validating algorithm performance. We present a novel stochastic framework based on a three-state hidden Markov model (HMM) (representing interictal, preictal, and seizure states) with the feature that periods of increased seizure probability can transition back to the interictal state. This notion reflects clinical experience and may enhance interpretation of published seizure prediction studies. Our model accommodates clipped EEG segments and formalizes intuitive notions regarding statistical validation. We derive equations for type I and type II errors as a function of the number of seizures, duration of interictal data, and prediction horizon length and we demonstrate the model’s utility with a novel seizure detection algorithm that appeared to predicted seizure onset. We propose this framework as a vital tool for designing and validating prediction algorithms and for facilitating collaborative research in this area. PMID:17021032

The Cost-Effectiveness of Lowering Permissible Noise Levels Around U.S. Airports

PubMed Central

Jiao, Boshen; Zafari, Zafar; Will, Brian; Ruggeri, Kai

2017-01-01

Aircraft noise increases the risk of cardiovascular diseases and mental illness. The allowable limit for sound in the vicinity of an airport is 65 decibels (dB) averaged over a 24-h ‘day and night’ period (DNL) in the United States. We evaluate the trade-off between the cost and the health benefits of changing the regulatory DNL level from 65 dB to 55 dB using a Markov model. The study used LaGuardia Airport (LGA) as a case study. In compliance with 55 dB allowable limit of aircraft noise, sound insulation would be required for residential homes within the 55 dB to 65 dB DNL. A Markov model was built to assess the cost-effectiveness of installing sound insulation. One-way sensitivity analyses and Monte Carlo simulation were conducted to test uncertainty of the model. The incremental cost-effectiveness ratio of installing sound insulation for residents exposed to airplane noise from LGA was $11,163/QALY gained (95% credible interval: cost-saving and life-saving to $93,054/QALY gained). Changing the regulatory standard for noise exposure around airports from 65 dB to 55 dB comes at a very good value. PMID:29207473

The Cost-Effectiveness of Lowering Permissible Noise Levels Around U.S. Airports.

PubMed

Jiao, Boshen; Zafari, Zafar; Will, Brian; Ruggeri, Kai; Li, Shukai; Muennig, Peter

2017-12-02

Aircraft noise increases the risk of cardiovascular diseases and mental illness. The allowable limit for sound in the vicinity of an airport is 65 decibels (dB) averaged over a 24-h 'day and night' period (DNL) in the United States. We evaluate the trade-off between the cost and the health benefits of changing the regulatory DNL level from 65 dB to 55 dB using a Markov model. The study used LaGuardia Airport (LGA) as a case study. In compliance with 55 dB allowable limit of aircraft noise, sound insulation would be required for residential homes within the 55 dB to 65 dB DNL. A Markov model was built to assess the cost-effectiveness of installing sound insulation. One-way sensitivity analyses and Monte Carlo simulation were conducted to test uncertainty of the model. The incremental cost-effectiveness ratio of installing sound insulation for residents exposed to airplane noise from LGA was $11,163/QALY gained (95% credible interval: cost-saving and life-saving to $93,054/QALY gained). Changing the regulatory standard for noise exposure around airports from 65 dB to 55 dB comes at a very good value.

AEOLUS: A MARKOV CHAIN MONTE CARLO CODE FOR MAPPING ULTRACOOL ATMOSPHERES. AN APPLICATION ON JUPITER AND BROWN DWARF HST LIGHT CURVES

DOE Office of Scientific and Technical Information (OSTI.GOV)

Karalidi, Theodora; Apai, Dániel; Schneider, Glenn

Deducing the cloud cover and its temporal evolution from the observed planetary spectra and phase curves can give us major insight into the atmospheric dynamics. In this paper, we present Aeolus, a Markov chain Monte Carlo code that maps the structure of brown dwarf and other ultracool atmospheres. We validated Aeolus on a set of unique Jupiter Hubble Space Telescope (HST) light curves. Aeolus accurately retrieves the properties of the major features of the Jovian atmosphere, such as the Great Red Spot and a major 5 μm hot spot. Aeolus is the first mapping code validated on actual observations of amore » giant planet over a full rotational period. For this study, we applied Aeolus to J- and H-band HST light curves of 2MASS J21392676+0220226 and 2MASS J0136565+093347. Aeolus retrieves three spots at the top of the atmosphere (per observational wavelength) of these two brown dwarfs, with a surface coverage of 21% ± 3% and 20.3% ± 1.5%, respectively. The Jupiter HST light curves will be publicly available via ADS/VIZIR.« less

A tridiagonal parsimonious higher order multivariate Markov chain model

NASA Astrophysics Data System (ADS)

Wang, Chao; Yang, Chuan-sheng

2017-09-01

In this paper, we present a tridiagonal parsimonious higher-order multivariate Markov chain model (TPHOMMCM). Moreover, estimation method of the parameters in TPHOMMCM is give. Numerical experiments illustrate the effectiveness of TPHOMMCM.

Reliability Assessment of Reconfigurable Flight Control Systems Using Sure and Assist

NASA Technical Reports Server (NTRS)

Wu, N. Eva

1992-01-01

This paper presents a reliability assessment of Reconfigurable Flight Control Systems using Semi-Markov Unreliability Range Evaluator (SURE) and Abstract Semi-Markov Specification Interface to the SURE Tool (ASSIST).

The application of Markov decision process with penalty function in restaurant delivery robot

NASA Astrophysics Data System (ADS)

Wang, Yong; Hu, Zhen; Wang, Ying

2017-05-01

As the restaurant delivery robot is often in a dynamic and complex environment, including the chairs inadvertently moved to the channel and customers coming and going. The traditional Markov decision process path planning algorithm is not save, the robot is very close to the table and chairs. To solve this problem, this paper proposes the Markov Decision Process with a penalty term called MDPPT path planning algorithm according to the traditional Markov decision process (MDP). For MDP, if the restaurant delivery robot bumps into an obstacle, the reward it receives is part of the current status reward. For the MDPPT, the reward it receives not only the part of the current status but also a negative constant term. Simulation results show that the MDPPT algorithm can plan a more secure path.

Three real-time architectures - A study using reward models

NASA Technical Reports Server (NTRS)

Sjogren, J. A.; Smith, R. M.

1990-01-01

Numerous applications in the area of computer system analysis can be effectively studied with Markov reward models. These models describe the evolutionary behavior of the computer system by a continuous-time Markov chain, and a reward rate is associated with each state. In reliability/availability models, upstates have reward rate 1, and down states have reward rate zero associated with them. In a combined model of performance and reliability, the reward rate of a state may be the computational capacity, or a related performance measure. Steady-state expected reward rate and expected instantaneous reward rate are clearly useful measures which can be extracted from the Markov reward model. The diversity of areas where Markov reward models may be used is illustrated with a comparative study of three examples of interest to the fault tolerant computing community.

Markov reward processes

NASA Technical Reports Server (NTRS)

Smith, R. M.

1991-01-01

Numerous applications in the area of computer system analysis can be effectively studied with Markov reward models. These models describe the behavior of the system with a continuous-time Markov chain, where a reward rate is associated with each state. In a reliability/availability model, upstates may have reward rate 1 and down states may have reward rate zero associated with them. In a queueing model, the number of jobs of certain type in a given state may be the reward rate attached to that state. In a combined model of performance and reliability, the reward rate of a state may be the computational capacity, or a related performance measure. Expected steady-state reward rate and expected instantaneous reward rate are clearly useful measures of the Markov reward model. More generally, the distribution of accumulated reward or time-averaged reward over a finite time interval may be determined from the solution of the Markov reward model. This information is of great practical significance in situations where the workload can be well characterized (deterministically, or by continuous functions e.g., distributions). The design process in the development of a computer system is an expensive and long term endeavor. For aerospace applications the reliability of the computer system is essential, as is the ability to complete critical workloads in a well defined real time interval. Consequently, effective modeling of such systems must take into account both performance and reliability. This fact motivates our use of Markov reward models to aid in the development and evaluation of fault tolerant computer systems.

A reward semi-Markov process with memory for wind speed modeling

NASA Astrophysics Data System (ADS)

Petroni, F.; D'Amico, G.; Prattico, F.

2012-04-01

The increasing interest in renewable energy leads scientific research to find a better way to recover most of the available energy. Particularly, the maximum energy recoverable from wind is equal to 59.3% of that available (Betz law) at a specific pitch angle and when the ratio between the wind speed in output and in input is equal to 1/3. The pitch angle is the angle formed between the airfoil of the blade of the wind turbine and the wind direction. Old turbine and a lot of that actually marketed, in fact, have always the same invariant geometry of the airfoil. This causes that wind turbines will work with an efficiency that is lower than 59.3%. New generation wind turbines, instead, have a system to variate the pitch angle by rotating the blades. This system able the wind turbines to recover, at different wind speed, always the maximum energy, working in Betz limit at different speed ratios. A powerful system control of the pitch angle allows the wind turbine to recover better the energy in transient regime. A good stochastic model for wind speed is then needed to help both the optimization of turbine design and to assist the system control to predict the value of the wind speed to positioning the blades quickly and correctly. The possibility to have synthetic data of wind speed is a powerful instrument to assist designer to verify the structures of the wind turbines or to estimate the energy recoverable from a specific site. To generate synthetic data, Markov chains of first or higher order are often used [1,2,3]. In particular in [1] is presented a comparison between a first-order Markov chain and a second-order Markov chain. A similar work, but only for the first-order Markov chain, is conduced by [2], presenting the probability transition matrix and comparing the energy spectral density and autocorrelation of real and synthetic wind speed data. A tentative to modeling and to join speed and direction of wind is presented in [3], by using two models, first-order Markov chain with different number of states, and Weibull distribution. All this model use Markov chains to generate synthetic wind speed time series but the search for a better model is still open. Approaching this issue, we applied new models which are generalization of Markov models. More precisely we applied semi-Markov models to generate synthetic wind speed time series. The primary goal of this analysis is the study of the time history of the wind in order to assess its reliability as a source of power and to determine the associated storage levels required. In order to assess this issue we use a probabilistic model based on indexed semi-Markov process [4] to which a reward structure is attached. Our model is used to calculate the expected energy produced by a given turbine and its variability expressed by the variance of the process. Our results can be used to compare different wind farms based on their reward and also on the risk of missed production due to the intrinsic variability of the wind speed process. The model is used to generate synthetic time series for wind speed by means of Monte Carlo simulations and backtesting procedure is used to compare results on first and second oder moments of rewards between real and synthetic data. [1] A. Shamshad, M.A. Bawadi, W.M.W. Wan Hussin, T.A. Majid, S.A.M. Sanusi, First and second order Markov chain models for synthetic gen- eration of wind speed time series, Energy 30 (2005) 693-708. [2] H. Nfaoui, H. Essiarab, A.A.M. Sayigh, A stochastic Markov chain model for simulating wind speed time series at Tangiers, Morocco, Re- newable Energy 29 (2004) 1407-1418. [3] F. Youcef Ettoumi, H. Sauvageot, A.-E.-H. Adane, Statistical bivariate modeling of wind using first-order Markov chain and Weibull distribu- tion, Renewable Energy 28 (2003) 1787-1802. [4]F. Petroni, G. D'Amico, F. Prattico, Indexed semi-Markov process for wind speed modeling. To be submitted.

A constraint-based evolutionary learning approach to the expectation maximization for optimal estimation of the hidden Markov model for speech signal modeling.

PubMed

Huda, Shamsul; Yearwood, John; Togneri, Roberto

2009-02-01

This paper attempts to overcome the tendency of the expectation-maximization (EM) algorithm to locate a local rather than global maximum when applied to estimate the hidden Markov model (HMM) parameters in speech signal modeling. We propose a hybrid algorithm for estimation of the HMM in automatic speech recognition (ASR) using a constraint-based evolutionary algorithm (EA) and EM, the CEL-EM. The novelty of our hybrid algorithm (CEL-EM) is that it is applicable for estimation of the constraint-based models with many constraints and large numbers of parameters (which use EM) like HMM. Two constraint-based versions of the CEL-EM with different fusion strategies have been proposed using a constraint-based EA and the EM for better estimation of HMM in ASR. The first one uses a traditional constraint-handling mechanism of EA. The other version transforms a constrained optimization problem into an unconstrained problem using Lagrange multipliers. Fusion strategies for the CEL-EM use a staged-fusion approach where EM has been plugged with the EA periodically after the execution of EA for a specific period of time to maintain the global sampling capabilities of EA in the hybrid algorithm. A variable initialization approach (VIA) has been proposed using a variable segmentation to provide a better initialization for EA in the CEL-EM. Experimental results on the TIMIT speech corpus show that CEL-EM obtains higher recognition accuracies than the traditional EM algorithm as well as a top-standard EM (VIA-EM, constructed by applying the VIA to EM).

Population Synthesis of Radio & Gamma-Ray Millisecond Pulsars

NASA Astrophysics Data System (ADS)

Frederick, Sara; Gonthier, P. L.; Harding, A. K.

2014-01-01

In recent years, the number of known gamma-ray millisecond pulsars (MSPs) in the Galactic disk has risen substantially thanks to confirmed detections by Fermi Gamma-ray Space Telescope (Fermi). We have developed a new population synthesis of gamma-ray and radio MSPs in the galaxy which uses Markov Chain Monte Carlo techniques to explore the large and small worlds of the model parameter space and allows for comparisons of the simulated and detected MSP distributions. The simulation employs empirical radio and gamma-ray luminosity models that are dependent upon the pulsar period and period derivative with freely varying exponents. Parameters associated with the birth distributions are also free to vary. The computer code adjusts the magnitudes of the model luminosities to reproduce the number of MSPs detected by a group of ten radio surveys, thus normalizing the simulation and predicting the MSP birth rates in the Galaxy. Computing many Markov chains leads to preferred sets of model parameters that are further explored through two statistical methods. Marginalized plots define confidence regions in the model parameter space using maximum likelihood methods. A secondary set of confidence regions is determined in parallel using Kuiper statistics calculated from comparisons of cumulative distributions. These two techniques provide feedback to affirm the results and to check for consistency. Radio flux and dispersion measure constraints have been imposed on the simulated gamma-ray distributions in order to reproduce realistic detection conditions. The simulated and detected distributions agree well for both sets of radio and gamma-ray pulsar characteristics, as evidenced by our various comparisons.

Mode-coupling of interaction quenched ultracold bosons in periodically driven lattices

NASA Astrophysics Data System (ADS)

Mistakidis, Simeon; Schmelcher, Peter

2016-05-01

The out-of-equilibrium dynamics of interaction quenched finite ultracold bosonic ensembles in periodically driven one-dimensional optical lattices is investigated. As a first attempt a brief analysis of the dynamics caused exclusively by the periodically driven lattice is presented and the induced low-lying modes are introduced. It is shown that the periodic driving enforces the bosons in the outer wells to exhibit out-of-phase dipole-like modes, while in the central well the cloud experiences a local-breathing mode. The dynamical behavior of the system is investigated with respect to the driving frequency, revealing a resonant-like behavior of the intra-well dynamics. Subsequently, we drive the system to a highly non-equilibrium state by performing an interaction quench upon the periodically driven lattice. This protocol gives rise to admixtures of excitations in the outer wells, an enhanced breathing in the center and an amplification of the tunneling dynamics. As a result (of the quench) the system experiences multiple resonances between the inter- and intra-well dynamics at different quench amplitudes. Finally, our study reveals that the position of the resonances can be adjusted e.g. via the driving frequency or the atom number manifesting their many-body nature. Deutsche Forschungsgemeinschaft (DFG) in the framework of the SFB 925 ``Light induced dynamics and control of correlated quantum systems''.

Markov chain model for demersal fish catch analysis in Indonesia

NASA Astrophysics Data System (ADS)

Firdaniza; Gusriani, N.

2018-03-01

As an archipelagic country, Indonesia has considerable potential fishery resources. One of the fish resources that has high economic value is demersal fish. Demersal fish is a fish with a habitat in the muddy seabed. Demersal fish scattered throughout the Indonesian seas. Demersal fish production in each Indonesia’s Fisheries Management Area (FMA) varies each year. In this paper we have discussed the Markov chain model for demersal fish yield analysis throughout all Indonesia’s Fisheries Management Area. Data of demersal fish catch in every FMA in 2005-2014 was obtained from Directorate of Capture Fisheries. From this data a transition probability matrix is determined by the number of transitions from the catch that lie below the median or above the median. The Markov chain model of demersal fish catch data was an ergodic Markov chain model, so that the limiting probability of the Markov chain model can be determined. The predictive value of demersal fishing yields was obtained by calculating the combination of limiting probability with average catch results below the median and above the median. The results showed that for 2018 and long-term demersal fishing results in most of FMA were below the median value.

Casting a Wider Net: Data Driven Discovery of Proxies for Target Diagnoses

PubMed Central

Ramljak, Dusan; Davey, Adam; Uversky, Alexey; Roychoudhury, Shoumik; Obradovic, Zoran

2015-01-01

Background: The Hospital Readmissions Reduction Program (HRRP) introduced in October 2012 as part of the Affordable Care Act (ACA), ties hospital reimbursement rates to adjusted 30-day readmissions and mortality performance for a small set of target diagnoses. There is growing concern and emerging evidence that use of a small set of target diagnoses to establish reimbursement rates can lead to unstable results that are susceptible to manipulation (gaming) by hospitals. Methods: We propose a novel approach to identifying co-occurring diagnoses and procedures that can themselves serve as a proxy indicator of the target diagnosis. The proposed approach constructs a Markov Blanket that allows a high level of performance, in terms of predictive accuracy and scalability, along with interpretability of obtained results. In order to scale to a large number of co-occuring diagnoses (features) and hospital discharge records (samples), our approach begins with Google’s PageRank algorithm and exploits the stability of obtained results to rank the contribution of each diagnosis/procedure in terms of presence in a Markov Blanket for outcome prediction. Results: Presence of target diagnoses acute myocardial infarction (AMI), congestive heart failure (CHF), pneumonia (PN), and Sepsis in hospital discharge records for Medicare and Medicaid patients in California and New York state hospitals (2009–2011), were predicted using models trained on a subset of California state hospitals (2003–2008). Using repeated holdout evaluation, we used ~30,000,000 hospital discharge records and analyzed the stability of the proposed approach. Model performance was measured using the Area Under the ROC Curve (AUC) metric, and importance and contribution of single features to the final result. The results varied from AUC=0.68 (with SE<1e-4) for PN on cross validation datasets to AUC=0.94, with (SE<1e-7) for Sepsis on California hospitals (2009 – 2011), while the stability of features was consistently better with more training data for each target diagnosis. Prediction accuracy for considered target diagnoses approaches or exceeds accuracy estimates for discharge record data. Conclusions: This paper presents a novel approach to identifying a small subset of relevant diagnoses and procedures that approximate the Markov Blanket for target diagnoses. Accuracy and interpretability of results demonstrate the potential of our approach. PMID:26958243

Analysis and design of a second-order digital phase-locked loop

NASA Technical Reports Server (NTRS)

Blasche, P. R.

1979-01-01

A specific second-order digital phase-locked loop (DPLL) was modeled as a first-order Markov chain with alternatives. From the matrix of transition probabilities of the Markov chain, the steady-state phase error of the DPLL was determined. In a similar manner the loop's response was calculated for a fading input. Additionally, a hardware DPLL was constructed and tested to provide a comparison to the results obtained from the Markov chain model. In all cases tested, good agreement was found between the theoretical predictions and the experimental data.

Reliability Analysis of the Electrical Control System of Subsea Blowout Preventers Using Markov Models

PubMed Central

Liu, Zengkai; Liu, Yonghong; Cai, Baoping

2014-01-01

Reliability analysis of the electrical control system of a subsea blowout preventer (BOP) stack is carried out based on Markov method. For the subsea BOP electrical control system used in the current work, the 3-2-1-0 and 3-2-0 input voting schemes are available. The effects of the voting schemes on system performance are evaluated based on Markov models. In addition, the effects of failure rates of the modules and repair time on system reliability indices are also investigated. PMID:25409010

Information Entropy Production of Maximum Entropy Markov Chains from Spike Trains

NASA Astrophysics Data System (ADS)

Cofré, Rodrigo; Maldonado, Cesar

2018-01-01

We consider the maximum entropy Markov chain inference approach to characterize the collective statistics of neuronal spike trains, focusing on the statistical properties of the inferred model. We review large deviations techniques useful in this context to describe properties of accuracy and convergence in terms of sampling size. We use these results to study the statistical fluctuation of correlations, distinguishability and irreversibility of maximum entropy Markov chains. We illustrate these applications using simple examples where the large deviation rate function is explicitly obtained for maximum entropy models of relevance in this field.

Closed-loop suppression of chaos in nonlinear driven oscillators

NASA Astrophysics Data System (ADS)

Aguirre, L. A.; Billings, S. A.

1995-05-01

This paper discusses the suppression of chaos in nonlinear driven oscillators via the addition of a periodic perturbation. Given a system originally undergoing chaotic motions, it is desired that such a system be driven to some periodic orbit. This can be achieved by the addition of a weak periodic signal to the oscillator input. This is usually accomplished in open loop, but this procedure presents some difficulties which are discussed in the paper. To ensure that this is attained despite uncertainties and possible disturbances on the system, a procedure is suggested to perform control in closed loop. In addition, it is illustrated how a model, estimated from input/output data, can be used in the design. Numerical examples which use the Duffing-Ueda and modified van der Pol oscillators are included to illustrate some of the properties of the new approach.

Marathon: An Open Source Software Library for the Analysis of Markov-Chain Monte Carlo Algorithms

PubMed Central

Rechner, Steffen; Berger, Annabell

2016-01-01

We present the software library marathon, which is designed to support the analysis of sampling algorithms that are based on the Markov-Chain Monte Carlo principle. The main application of this library is the computation of properties of so-called state graphs, which represent the structure of Markov chains. We demonstrate applications and the usefulness of marathon by investigating the quality of several bounding methods on four well-known Markov chains for sampling perfect matchings and bipartite graphs. In a set of experiments, we compute the total mixing time and several of its bounds for a large number of input instances. We find that the upper bound gained by the famous canonical path method is often several magnitudes larger than the total mixing time and deteriorates with growing input size. In contrast, the spectral bound is found to be a precise approximation of the total mixing time. PMID:26824442

A Langevin equation for the rates of currency exchange based on the Markov analysis

NASA Astrophysics Data System (ADS)

Farahpour, F.; Eskandari, Z.; Bahraminasab, A.; Jafari, G. R.; Ghasemi, F.; Sahimi, Muhammad; Reza Rahimi Tabar, M.

2007-11-01

We propose a method for analyzing the data for the rates of exchange of various currencies versus the U.S. dollar. The method analyzes the return time series of the data as a Markov process, and develops an effective equation which reconstructs it. We find that the Markov time scale, i.e., the time scale over which the data are Markov-correlated, is one day for the majority of the daily exchange rates that we analyze. We derive an effective Langevin equation to describe the fluctuations in the rates. The equation contains two quantities, D and D, representing the drift and diffusion coefficients, respectively. We demonstrate how the two coefficients are estimated directly from the data, without using any assumptions or models for the underlying stochastic time series that represent the daily rates of exchange of various currencies versus the U.S. dollar.

Decomposition of conditional probability for high-order symbolic Markov chains.

PubMed

Melnik, S S; Usatenko, O V

2017-07-01

The main goal of this paper is to develop an estimate for the conditional probability function of random stationary ergodic symbolic sequences with elements belonging to a finite alphabet. We elaborate on a decomposition procedure for the conditional probability function of sequences considered to be high-order Markov chains. We represent the conditional probability function as the sum of multilinear memory function monomials of different orders (from zero up to the chain order). This allows us to introduce a family of Markov chain models and to construct artificial sequences via a method of successive iterations, taking into account at each step increasingly high correlations among random elements. At weak correlations, the memory functions are uniquely expressed in terms of the high-order symbolic correlation functions. The proposed method fills the gap between two approaches, namely the likelihood estimation and the additive Markov chains. The obtained results may have applications for sequential approximation of artificial neural network training.

Monte Carlo estimation of total variation distance of Markov chains on large spaces, with application to phylogenetics.

PubMed

Herbei, Radu; Kubatko, Laura

2013-03-26

Markov chains are widely used for modeling in many areas of molecular biology and genetics. As the complexity of such models advances, it becomes increasingly important to assess the rate at which a Markov chain converges to its stationary distribution in order to carry out accurate inference. A common measure of convergence to the stationary distribution is the total variation distance, but this measure can be difficult to compute when the state space of the chain is large. We propose a Monte Carlo method to estimate the total variation distance that can be applied in this situation, and we demonstrate how the method can be efficiently implemented by taking advantage of GPU computing techniques. We apply the method to two Markov chains on the space of phylogenetic trees, and discuss the implications of our findings for the development of algorithms for phylogenetic inference.

A high-fidelity weather time series generator using the Markov Chain process on a piecewise level

NASA Astrophysics Data System (ADS)

Hersvik, K.; Endrerud, O.-E. V.

2017-12-01

A method is developed for generating a set of unique weather time-series based on an existing weather series. The method allows statistically valid weather variations to take place within repeated simulations of offshore operations. The numerous generated time series need to share the same statistical qualities as the original time series. Statistical qualities here refer mainly to the distribution of weather windows available for work, including durations and frequencies of such weather windows, and seasonal characteristics. The method is based on the Markov chain process. The core new development lies in how the Markov Process is used, specifically by joining small pieces of random length time series together rather than joining individual weather states, each from a single time step, which is a common solution found in the literature. This new Markov model shows favorable characteristics with respect to the requirements set forth and all aspects of the validation performed.

Decomposition of conditional probability for high-order symbolic Markov chains

NASA Astrophysics Data System (ADS)

Melnik, S. S.; Usatenko, O. V.

2017-07-01

The main goal of this paper is to develop an estimate for the conditional probability function of random stationary ergodic symbolic sequences with elements belonging to a finite alphabet. We elaborate on a decomposition procedure for the conditional probability function of sequences considered to be high-order Markov chains. We represent the conditional probability function as the sum of multilinear memory function monomials of different orders (from zero up to the chain order). This allows us to introduce a family of Markov chain models and to construct artificial sequences via a method of successive iterations, taking into account at each step increasingly high correlations among random elements. At weak correlations, the memory functions are uniquely expressed in terms of the high-order symbolic correlation functions. The proposed method fills the gap between two approaches, namely the likelihood estimation and the additive Markov chains. The obtained results may have applications for sequential approximation of artificial neural network training.

Weighted Markov chains for forecasting and analysis in Incidence of infectious diseases in jiangsu Province, China☆

PubMed Central

Peng, Zhihang; Bao, Changjun; Zhao, Yang; Yi, Honggang; Xia, Letian; Yu, Hao; Shen, Hongbing; Chen, Feng

2010-01-01

This paper first applies the sequential cluster method to set up the classification standard of infectious disease incidence state based on the fact that there are many uncertainty characteristics in the incidence course. Then the paper presents a weighted Markov chain, a method which is used to predict the future incidence state. This method assumes the standardized self-coefficients as weights based on the special characteristics of infectious disease incidence being a dependent stochastic variable. It also analyzes the characteristics of infectious diseases incidence via the Markov chain Monte Carlo method to make the long-term benefit of decision optimal. Our method is successfully validated using existing incidents data of infectious diseases in Jiangsu Province. In summation, this paper proposes ways to improve the accuracy of the weighted Markov chain, specifically in the field of infection epidemiology. PMID:23554632

Weighted Markov chains for forecasting and analysis in Incidence of infectious diseases in jiangsu Province, China.

PubMed

Peng, Zhihang; Bao, Changjun; Zhao, Yang; Yi, Honggang; Xia, Letian; Yu, Hao; Shen, Hongbing; Chen, Feng

2010-05-01

This paper first applies the sequential cluster method to set up the classification standard of infectious disease incidence state based on the fact that there are many uncertainty characteristics in the incidence course. Then the paper presents a weighted Markov chain, a method which is used to predict the future incidence state. This method assumes the standardized self-coefficients as weights based on the special characteristics of infectious disease incidence being a dependent stochastic variable. It also analyzes the characteristics of infectious diseases incidence via the Markov chain Monte Carlo method to make the long-term benefit of decision optimal. Our method is successfully validated using existing incidents data of infectious diseases in Jiangsu Province. In summation, this paper proposes ways to improve the accuracy of the weighted Markov chain, specifically in the field of infection epidemiology.

Canonical Structure and Orthogonality of Forces and Currents in Irreversible Markov Chains

NASA Astrophysics Data System (ADS)

Kaiser, Marcus; Jack, Robert L.; Zimmer, Johannes

2018-03-01

We discuss a canonical structure that provides a unifying description of dynamical large deviations for irreversible finite state Markov chains (continuous time), Onsager theory, and Macroscopic Fluctuation Theory (MFT). For Markov chains, this theory involves a non-linear relation between probability currents and their conjugate forces. Within this framework, we show how the forces can be split into two components, which are orthogonal to each other, in a generalised sense. This splitting allows a decomposition of the pathwise rate function into three terms, which have physical interpretations in terms of dissipation and convergence to equilibrium. Similar decompositions hold for rate functions at level 2 and level 2.5. These results clarify how bounds on entropy production and fluctuation theorems emerge from the underlying dynamical rules. We discuss how these results for Markov chains are related to similar structures within MFT, which describes hydrodynamic limits of such microscopic models.

VAMPnets for deep learning of molecular kinetics.

PubMed

Mardt, Andreas; Pasquali, Luca; Wu, Hao; Noé, Frank

2018-01-02

There is an increasing demand for computing the relevant structures, equilibria, and long-timescale kinetics of biomolecular processes, such as protein-drug binding, from high-throughput molecular dynamics simulations. Current methods employ transformation of simulated coordinates into structural features, dimension reduction, clustering the dimension-reduced data, and estimation of a Markov state model or related model of the interconversion rates between molecular structures. This handcrafted approach demands a substantial amount of modeling expertise, as poor decisions at any step will lead to large modeling errors. Here we employ the variational approach for Markov processes (VAMP) to develop a deep learning framework for molecular kinetics using neural networks, dubbed VAMPnets. A VAMPnet encodes the entire mapping from molecular coordinates to Markov states, thus combining the whole data processing pipeline in a single end-to-end framework. Our method performs equally or better than state-of-the-art Markov modeling methods and provides easily interpretable few-state kinetic models.

Neuron’s eye view: Inferring features of complex stimuli from neural responses

PubMed Central

Chen, Xin; Beck, Jeffrey M.

2017-01-01

Experiments that study neural encoding of stimuli at the level of individual neurons typically choose a small set of features present in the world—contrast and luminance for vision, pitch and intensity for sound—and assemble a stimulus set that systematically varies along these dimensions. Subsequent analysis of neural responses to these stimuli typically focuses on regression models, with experimenter-controlled features as predictors and spike counts or firing rates as responses. Unfortunately, this approach requires knowledge in advance about the relevant features coded by a given population of neurons. For domains as complex as social interaction or natural movement, however, the relevant feature space is poorly understood, and an arbitrary a priori choice of features may give rise to confirmation bias. Here, we present a Bayesian model for exploratory data analysis that is capable of automatically identifying the features present in unstructured stimuli based solely on neuronal responses. Our approach is unique within the class of latent state space models of neural activity in that it assumes that firing rates of neurons are sensitive to multiple discrete time-varying features tied to the stimulus, each of which has Markov (or semi-Markov) dynamics. That is, we are modeling neural activity as driven by multiple simultaneous stimulus features rather than intrinsic neural dynamics. We derive a fast variational Bayesian inference algorithm and show that it correctly recovers hidden features in synthetic data, as well as ground-truth stimulus features in a prototypical neural dataset. To demonstrate the utility of the algorithm, we also apply it to cluster neural responses and demonstrate successful recovery of features corresponding to monkeys and faces in the image set. PMID:28827790

Spin and topological order in a periodically driven spin chain

NASA Astrophysics Data System (ADS)

Russomanno, Angelo; Friedman, Bat-el; Dalla Torre, Emanuele G.

2017-07-01

The periodically driven quantum Ising chain has recently attracted a large attention in the context of Floquet engineering. In addition to the common paramagnet and ferromagnet, this driven model can give rise to new topological phases. In this work, we systematically explore its quantum phase diagram by examining the properties of its Floquet ground state. We specifically focus on driving protocols with time-reversal invariant points, and demonstrate the existence of an infinite number of distinct phases. These phases are separated by second-order quantum phase transitions, accompanied by continuous changes of local and string order parameters, as well as sudden changes of a topological winding number and of the number of protected edge states. When one of these phase transitions is adiabatically crossed, the correlator associated to the order parameter is nonvanishing over a length scale which shows a Kibble-Zurek scaling. In some phases, the Floquet ground state spontaneously breaks the discrete time-translation symmetry of the Hamiltonian. Our findings provide a better understanding of topological phases in periodically driven clean integrable models.

Chaos and Hyperchaos in Coupled Antiphase Driven Toda Oscillators

NASA Astrophysics Data System (ADS)

Stankevich, Nataliya V.; Dvorak, Anton; Astakhov, Vladimir; Jaros, Patrycja; Kapitaniak, Marcin; Perlikowski, Przemysław; Kapitaniak, Tomasz

2018-01-01

The dynamics of two coupled antiphase driven Toda oscillators is studied. We demonstrate three different routes of transition to chaotic dynamics associated with different bifurcations of periodic and quasi-periodic regimes. As a result of these, two types of chaotic dynamics with one and two positive Lyapunov exponents are observed. We argue that the results obtained are robust as they can exist in a wide range of the system parameters.

Data-Driven and Expectation-Driven Discovery of Empirical Laws.

DTIC Science & Technology

1982-10-10

occurred in small integer proportions to each other. In 1809, Joseph Gay- Lussac found evidence for his law of combining volumes, which stated that a...of Empirical Laws Patrick W. Langley Gary L. Bradshaw Herbert A. Simon T1he Robotics Institute Carnegie-Mellon University Pittsburgh, Pennsylvania...Subtitle) S. TYPE OF REPORT & PERIOD COVERED Data-Driven and Expectation-Driven Discovery Interim Report 2/82-10/82 of Empirical Laws S. PERFORMING ORG

Monte Carlo Simulation of Markov, Semi-Markov, and Generalized Semi- Markov Processes in Probabilistic Risk Assessment

NASA Technical Reports Server (NTRS)

English, Thomas

2005-01-01

A standard tool of reliability analysis used at NASA-JSC is the event tree. An event tree is simply a probability tree, with the probabilities determining the next step through the tree specified at each node. The nodal probabilities are determined by a reliability study of the physical system at work for a particular node. The reliability study performed at a node is typically referred to as a fault tree analysis, with the potential of a fault tree existing.for each node on the event tree. When examining an event tree it is obvious why the event tree/fault tree approach has been adopted. Typical event trees are quite complex in nature, and the event tree/fault tree approach provides a systematic and organized approach to reliability analysis. The purpose of this study was two fold. Firstly, we wanted to explore the possibility that a semi-Markov process can create dependencies between sojourn times (the times it takes to transition from one state to the next) that can decrease the uncertainty when estimating time to failures. Using a generalized semi-Markov model, we studied a four element reliability model and were able to demonstrate such sojourn time dependencies. Secondly, we wanted to study the use of semi-Markov processes to introduce a time variable into the event tree diagrams that are commonly developed in PRA (Probabilistic Risk Assessment) analyses. Event tree end states which change with time are more representative of failure scenarios than are the usual static probability-derived end states.

Parallel Markov chain Monte Carlo - bridging the gap to high-performance Bayesian computation in animal breeding and genetics.

PubMed

Wu, Xiao-Lin; Sun, Chuanyu; Beissinger, Timothy M; Rosa, Guilherme Jm; Weigel, Kent A; Gatti, Natalia de Leon; Gianola, Daniel

2012-09-25

Most Bayesian models for the analysis of complex traits are not analytically tractable and inferences are based on computationally intensive techniques. This is true of Bayesian models for genome-enabled selection, which uses whole-genome molecular data to predict the genetic merit of candidate animals for breeding purposes. In this regard, parallel computing can overcome the bottlenecks that can arise from series computing. Hence, a major goal of the present study is to bridge the gap to high-performance Bayesian computation in the context of animal breeding and genetics. Parallel Monte Carlo Markov chain algorithms and strategies are described in the context of animal breeding and genetics. Parallel Monte Carlo algorithms are introduced as a starting point including their applications to computing single-parameter and certain multiple-parameter models. Then, two basic approaches for parallel Markov chain Monte Carlo are described: one aims at parallelization within a single chain; the other is based on running multiple chains, yet some variants are discussed as well. Features and strategies of the parallel Markov chain Monte Carlo are illustrated using real data, including a large beef cattle dataset with 50K SNP genotypes. Parallel Markov chain Monte Carlo algorithms are useful for computing complex Bayesian models, which does not only lead to a dramatic speedup in computing but can also be used to optimize model parameters in complex Bayesian models. Hence, we anticipate that use of parallel Markov chain Monte Carlo will have a profound impact on revolutionizing the computational tools for genomic selection programs.

Parallel Markov chain Monte Carlo - bridging the gap to high-performance Bayesian computation in animal breeding and genetics

PubMed Central

2012-01-01

Background Most Bayesian models for the analysis of complex traits are not analytically tractable and inferences are based on computationally intensive techniques. This is true of Bayesian models for genome-enabled selection, which uses whole-genome molecular data to predict the genetic merit of candidate animals for breeding purposes. In this regard, parallel computing can overcome the bottlenecks that can arise from series computing. Hence, a major goal of the present study is to bridge the gap to high-performance Bayesian computation in the context of animal breeding and genetics. Results Parallel Monte Carlo Markov chain algorithms and strategies are described in the context of animal breeding and genetics. Parallel Monte Carlo algorithms are introduced as a starting point including their applications to computing single-parameter and certain multiple-parameter models. Then, two basic approaches for parallel Markov chain Monte Carlo are described: one aims at parallelization within a single chain; the other is based on running multiple chains, yet some variants are discussed as well. Features and strategies of the parallel Markov chain Monte Carlo are illustrated using real data, including a large beef cattle dataset with 50K SNP genotypes. Conclusions Parallel Markov chain Monte Carlo algorithms are useful for computing complex Bayesian models, which does not only lead to a dramatic speedup in computing but can also be used to optimize model parameters in complex Bayesian models. Hence, we anticipate that use of parallel Markov chain Monte Carlo will have a profound impact on revolutionizing the computational tools for genomic selection programs. PMID:23009363

Critical dynamics of gravito-convective mixing in geological carbon sequestration

DOE PAGES

Soltanian, Mohamad Reza; Amooie, Mohammad Amin; Dai, Zhenxue; ...

2016-11-03

When CO 2 is injected in saline aquifers, dissolution causes a local increase in brine density that can cause Rayleigh-Taylor-type gravitational instabilities. Depending on the Rayleigh number, density-driven flow may mix dissolved CO 2 throughout the aquifer at fast advective time-scales through convective mixing. Heterogeneity can impact density-driven flow to different degrees. Zones with low effective vertical permeability may suppress fingering and reduce vertical spreading, while potentially increasing transverse mixing. In more complex heterogeneity, arising from the spatial organization of sedimentary facies, finger propagation is reduced in low permeability facies, but may be enhanced through more permeable facies. The connectivitymore » of facies is critical in determining the large-scale transport of CO 2-rich brine. We perform high-resolution finite element simulations of advection-diffusion transport of CO 2 with a focus on facies-based bimodal heterogeneity. Permeability fields are generated by a Markov Chain approach, which represent facies architecture by commonly observed characteristics such as volume fractions. CO 2 dissolution and phase behavior are modeled with the cubic-plus-association equation-of-state. Our results show that the organization of high-permeability facies and their connectivity control the dynamics of gravitationally unstable flow. Lastly, we discover new flow regimes in both homogeneous and heterogeneous media and present quantitative scaling relations for their temporal evolution.« less

Probability Sampling Method for a Hidden Population Using Respondent-Driven Sampling: Simulation for Cancer Survivors.

PubMed

Jung, Minsoo

2015-01-01

When there is no sampling frame within a certain group or the group is concerned that making its population public would bring social stigma, we say the population is hidden. It is difficult to approach this kind of population survey-methodologically because the response rate is low and its members are not quite honest with their responses when probability sampling is used. The only alternative known to address the problems caused by previous methods such as snowball sampling is respondent-driven sampling (RDS), which was developed by Heckathorn and his colleagues. RDS is based on a Markov chain, and uses the social network information of the respondent. This characteristic allows for probability sampling when we survey a hidden population. We verified through computer simulation whether RDS can be used on a hidden population of cancer survivors. According to the simulation results of this thesis, the chain-referral sampling of RDS tends to minimize as the sample gets bigger, and it becomes stabilized as the wave progresses. Therefore, it shows that the final sample information can be completely independent from the initial seeds if a certain level of sample size is secured even if the initial seeds were selected through convenient sampling. Thus, RDS can be considered as an alternative which can improve upon both key informant sampling and ethnographic surveys, and it needs to be utilized for various cases domestically as well.

Markov Chain Estimation of Avian Seasonal Fecundity

EPA Science Inventory

To explore the consequences of modeling decisions on inference about avian seasonal fecundity we generalize previous Markov chain (MC) models of avian nest success to formulate two different MC models of avian seasonal fecundity that represent two different ways to model renestin...

Nonparametric model validations for hidden Markov models with applications in financial econometrics.

PubMed

Zhao, Zhibiao

2011-06-01

We address the nonparametric model validation problem for hidden Markov models with partially observable variables and hidden states. We achieve this goal by constructing a nonparametric simultaneous confidence envelope for transition density function of the observable variables and checking whether the parametric density estimate is contained within such an envelope. Our specification test procedure is motivated by a functional connection between the transition density of the observable variables and the Markov transition kernel of the hidden states. Our approach is applicable for continuous time diffusion models, stochastic volatility models, nonlinear time series models, and models with market microstructure noise.

MC3: Multi-core Markov-chain Monte Carlo code

NASA Astrophysics Data System (ADS)

Cubillos, Patricio; Harrington, Joseph; Lust, Nate; Foster, AJ; Stemm, Madison; Loredo, Tom; Stevenson, Kevin; Campo, Chris; Hardin, Matt; Hardy, Ryan

2016-10-01

MC3 (Multi-core Markov-chain Monte Carlo) is a Bayesian statistics tool that can be executed from the shell prompt or interactively through the Python interpreter with single- or multiple-CPU parallel computing. It offers Markov-chain Monte Carlo (MCMC) posterior-distribution sampling for several algorithms, Levenberg-Marquardt least-squares optimization, and uniform non-informative, Jeffreys non-informative, or Gaussian-informative priors. MC3 can share the same value among multiple parameters and fix the value of parameters to constant values, and offers Gelman-Rubin convergence testing and correlated-noise estimation with time-averaging or wavelet-based likelihood estimation methods.

Markov Chain Model with Catastrophe to Determine Mean Time to Default of Credit Risky Assets

NASA Astrophysics Data System (ADS)

Dharmaraja, Selvamuthu; Pasricha, Puneet; Tardelli, Paola

2017-11-01

This article deals with the problem of probabilistic prediction of the time distance to default for a firm. To model the credit risk, the dynamics of an asset is described as a function of a homogeneous discrete time Markov chain subject to a catastrophe, the default. The behaviour of the Markov chain is investigated and the mean time to the default is expressed in a closed form. The methodology to estimate the parameters is given. Numerical results are provided to illustrate the applicability of the proposed model on real data and their analysis is discussed.

A Markov Chain-based quantitative study of angular distribution of photons through turbid slabs via isotropic light scattering

NASA Astrophysics Data System (ADS)

Li, Xuesong; Northrop, William F.

2016-04-01

This paper describes a quantitative approach to approximate multiple scattering through an isotropic turbid slab based on Markov Chain theorem. There is an increasing need to utilize multiple scattering for optical diagnostic purposes; however, existing methods are either inaccurate or computationally expensive. Here, we develop a novel Markov Chain approximation approach to solve multiple scattering angular distribution (AD) that can accurately calculate AD while significantly reducing computational cost compared to Monte Carlo simulation. We expect this work to stimulate ongoing multiple scattering research and deterministic reconstruction algorithm development with AD measurements.

Markov Analysis of Sleep Dynamics

NASA Astrophysics Data System (ADS)

Kim, J. W.; Lee, J.-S.; Robinson, P. A.; Jeong, D.-U.

2009-05-01

A new approach, based on a Markov transition matrix, is proposed to explain frequent sleep and wake transitions during sleep. The matrix is determined by analyzing hypnograms of 113 obstructive sleep apnea patients. Our approach shows that the statistics of sleep can be constructed via a single Markov process and that durations of all states have modified exponential distributions, in contrast to recent reports of a scale-free form for the wake stage and an exponential form for the sleep stage. Hypnograms of the same subjects, but treated with Continuous Positive Airway Pressure, are analyzed and compared quantitatively with the pretreatment ones, suggesting potential clinical applications.

Forecasting land-cover growth using remotely sensed data: a case study of the Igneada protection area in Turkey.

PubMed

Bozkaya, A Gonca; Balcik, Filiz Bektas; Goksel, Cigdem; Esbah, Hayriye

2015-03-01

Human activities in many parts of the world have greatly affected natural areas. Therefore, monitoring and forecasting of land-cover changes are important components for sustainable utilization, conservation, and development of these areas. This research has been conducted on Igneada, a legally protected area on the northwest coast of Turkey, which is famous for its unique, mangrove forests. The main focus of this study was to apply a land use and cover model that could quantitatively and graphically present the changes and its impacts on Igneada landscapes in the future. In this study, a Markov chain-based, stochastic Markov model and cellular automata Markov model were used. These models were calibrated using a time series of developed areas derived from Landsat Thematic Mapper (TM) imagery between 1990 and 2010 that also projected future growth to 2030. The results showed that CA Markov yielded reliable information better than St. Markov model. The findings displayed constant but overall slight increase of settlement and forest cover, and slight decrease of agricultural lands. However, even the slightest unsustainable change can put a significant pressure on the sensitive ecosystems of Igneada. Therefore, the management of the protected area should not only focus on the landscape composition but also pay attention to landscape configuration.

Exact goodness-of-fit tests for Markov chains.

PubMed

Besag, J; Mondal, D

2013-06-01

Goodness-of-fit tests are useful in assessing whether a statistical model is consistent with available data. However, the usual χ² asymptotics often fail, either because of the paucity of the data or because a nonstandard test statistic is of interest. In this article, we describe exact goodness-of-fit tests for first- and higher order Markov chains, with particular attention given to time-reversible ones. The tests are obtained by conditioning on the sufficient statistics for the transition probabilities and are implemented by simple Monte Carlo sampling or by Markov chain Monte Carlo. They apply both to single and to multiple sequences and allow a free choice of test statistic. Three examples are given. The first concerns multiple sequences of dry and wet January days for the years 1948-1983 at Snoqualmie Falls, Washington State, and suggests that standard analysis may be misleading. The second one is for a four-state DNA sequence and lends support to the original conclusion that a second-order Markov chain provides an adequate fit to the data. The last one is six-state atomistic data arising in molecular conformational dynamics simulation of solvated alanine dipeptide and points to strong evidence against a first-order reversible Markov chain at 6 picosecond time steps. © 2013, The International Biometric Society.

Invited commentary: Lost in estimation--searching for alternatives to markov chains to fit complex Bayesian models.

PubMed

Molitor, John

2012-03-01

Bayesian methods have seen an increase in popularity in a wide variety of scientific fields, including epidemiology. One of the main reasons for their widespread application is the power of the Markov chain Monte Carlo (MCMC) techniques generally used to fit these models. As a result, researchers often implicitly associate Bayesian models with MCMC estimation procedures. However, Bayesian models do not always require Markov-chain-based methods for parameter estimation. This is important, as MCMC estimation methods, while generally quite powerful, are complex and computationally expensive and suffer from convergence problems related to the manner in which they generate correlated samples used to estimate probability distributions for parameters of interest. In this issue of the Journal, Cole et al. (Am J Epidemiol. 2012;175(5):368-375) present an interesting paper that discusses non-Markov-chain-based approaches to fitting Bayesian models. These methods, though limited, can overcome some of the problems associated with MCMC techniques and promise to provide simpler approaches to fitting Bayesian models. Applied researchers will find these estimation approaches intuitively appealing and will gain a deeper understanding of Bayesian models through their use. However, readers should be aware that other non-Markov-chain-based methods are currently in active development and have been widely published in other fields.

Markov switching of the electricity supply curve and power prices dynamics

NASA Astrophysics Data System (ADS)

Mari, Carlo; Cananà, Lucianna

2012-02-01

Regime-switching models seem to well capture the main features of power prices behavior in deregulated markets. In a recent paper, we have proposed an equilibrium methodology to derive electricity prices dynamics from the interplay between supply and demand in a stochastic environment. In particular, assuming that the supply function is described by a power law where the exponent is a two-state strictly positive Markov process, we derived a regime switching dynamics of power prices in which regime switches are induced by transitions between Markov states. In this paper, we provide a dynamical model to describe the random behavior of power prices where the only non-Brownian component of the motion is endogenously introduced by Markov transitions in the exponent of the electricity supply curve. In this context, the stochastic process driving the switching mechanism becomes observable, and we will show that the non-Brownian component of the dynamics induced by transitions from Markov states is responsible for jumps and spikes of very high magnitude. The empirical analysis performed on three Australian markets confirms that the proposed approach seems quite flexible and capable of incorporating the main features of power prices time-series, thus reproducing the first four moments of log-returns empirical distributions in a satisfactory way.

Free energies from dynamic weighted histogram analysis using unbiased Markov state model.

PubMed

Rosta, Edina; Hummer, Gerhard

2015-01-13

The weighted histogram analysis method (WHAM) is widely used to obtain accurate free energies from biased molecular simulations. However, WHAM free energies can exhibit significant errors if some of the biasing windows are not fully equilibrated. To account for the lack of full equilibration, we develop the dynamic histogram analysis method (DHAM). DHAM uses a global Markov state model to obtain the free energy along the reaction coordinate. A maximum likelihood estimate of the Markov transition matrix is constructed by joint unbiasing of the transition counts from multiple umbrella-sampling simulations along discretized reaction coordinates. The free energy profile is the stationary distribution of the resulting Markov matrix. For this matrix, we derive an explicit approximation that does not require the usual iterative solution of WHAM. We apply DHAM to model systems, a chemical reaction in water treated using quantum-mechanics/molecular-mechanics (QM/MM) simulations, and the Na(+) ion passage through the membrane-embedded ion channel GLIC. We find that DHAM gives accurate free energies even in cases where WHAM fails. In addition, DHAM provides kinetic information, which we here use to assess the extent of convergence in each of the simulation windows. DHAM may also prove useful in the construction of Markov state models from biased simulations in phase-space regions with otherwise low population.

Dynamical quantum phase transitions in discrete time crystals

NASA Astrophysics Data System (ADS)

Kosior, Arkadiusz; Sacha, Krzysztof

2018-05-01

Discrete time crystals are related to nonequilibrium dynamics of periodically driven quantum many-body systems where the discrete time-translation symmetry of the Hamiltonian is spontaneously broken into another discrete symmetry. Recently, the concept of phase transitions has been extended to nonequilibrium dynamics of time-independent systems induced by a quantum quench, i.e., a sudden change of some parameter of the Hamiltonian. There, the return probability of a system to the ground state reveals singularities in time which are dubbed dynamical quantum phase transitions. We show that the quantum quench in a discrete time crystal leads to dynamical quantum phase transitions where the return probability of a periodically driven system to a Floquet eigenstate before the quench reveals singularities in time. It indicates that dynamical quantum phase transitions are not restricted to time-independent systems and can be also observed in systems that are periodically driven. We discuss how the phenomenon can be observed in ultracold atomic gases.

Simulation of high-energy radiation belt electron fluxes using NARMAX-VERB coupled codes

PubMed Central

Pakhotin, I P; Drozdov, A Y; Shprits, Y Y; Boynton, R J; Subbotin, D A; Balikhin, M A

2014-01-01

This study presents a fusion of data-driven and physics-driven methodologies of energetic electron flux forecasting in the outer radiation belt. Data-driven NARMAX (Nonlinear AutoRegressive Moving Averages with eXogenous inputs) model predictions for geosynchronous orbit fluxes have been used as an outer boundary condition to drive the physics-based Versatile Electron Radiation Belt (VERB) code, to simulate energetic electron fluxes in the outer radiation belt environment. The coupled system has been tested for three extended time periods totalling several weeks of observations. The time periods involved periods of quiet, moderate, and strong geomagnetic activity and captured a range of dynamics typical of the radiation belts. The model has successfully simulated energetic electron fluxes for various magnetospheric conditions. Physical mechanisms that may be responsible for the discrepancies between the model results and observations are discussed. PMID:26167432

Rainfall extremes, weather and climatic characterization over complex terrain: A data-driven approach based on signal enhancement methods and extreme value modeling

NASA Astrophysics Data System (ADS)

Pineda, Luis E.; Willems, Patrick

2017-04-01

Weather and climatic characterization of rainfall extremes is both of scientific and societal value for hydrometeorogical risk management, yet discrimination of local and large-scale forcing remains challenging in data-scarce and complex terrain environments. Here, we present an analysis framework that separate weather (seasonal) regimes and climate (inter-annual) influences using data-driven process identification. The approach is based on signal-to-noise separation methods and extreme value (EV) modeling of multisite rainfall extremes. The EV models use a semi-automatic parameter learning [1] for model identification across temporal scales. At weather scale, the EV models are combined with a state-based hidden Markov model [2] to represent the spatio-temporal structure of rainfall as persistent weather states. At climatic scale, the EV models are used to decode the drivers leading to the shift of weather patterns. The decoding is performed into a climate-to-weather signal subspace, built via dimension reduction of climate model proxies (e.g. sea surface temperature and atmospheric circulation) We apply the framework to the Western Andean Ridge (WAR) in Ecuador and Peru (0-6°S) using ground data from the second half of the 20th century. We find that the meridional component of winds is what matters for the in-year and inter-annual variability of high rainfall intensities alongside the northern WAR (0-2.5°S). There, low-level southerly winds are found as advection drivers for oceanic moist of the normal-rainy season and weak/moderate the El Niño (EN) type; but, the strong EN type and its unique moisture surplus is locally advected at lowlands in the central WAR. Moreover, the coastal ridges, south of 3°S dampen meridional airflows, leaving local hygrothermal gradients to control the in-year distribution of rainfall extremes and their anomalies. Overall, we show that the framework, which does not make any prior assumption on the explanatory power of the weather and climate drivers, allows identification of well-known features of the regional climate in a purely data-driven fashion. Thus, this approach shows potential for characterization of precipitation extremes in data-scarce and orographically complex regions in which model reconstructions are the only climate proxies References [1] Mínguez, R., F.J. Méndez, C. Izaguirre, M. Menéndez, and I.J. Losada (2010), Pseudooptimal parameter selection of non-stationary generalized extreme value models for environmental variables, Environ. Modell. Softw. 25, 1592-1607. [2] Pineda, L., P. Willems (2016), Multisite Downscaling of Seasonal Predictions to Daily Rainfall Characteristics over Pacific-Andean River Basins in Ecuador and Peru using a non-homogenous hidden Markov model, J. Hydrometeor, 17(2), 481-498, doi:10.1175/JHM-D-15-0040.1, http://journals.ametsoc.org/doi/full/10.1175/JHM-D-15-0040.1

Numerical research of the optimal control problem in the semi-Markov inventory model

DOE Office of Scientific and Technical Information (OSTI.GOV)

Gorshenin, Andrey K.; Belousov, Vasily V.; Shnourkoff, Peter V.

2015-03-10

This paper is devoted to the numerical simulation of stochastic system for inventory management products using controlled semi-Markov process. The results of a special software for the system’s research and finding the optimal control are presented.

Stochastic entrainment of a stochastic oscillator.

PubMed

Wang, Guanyu; Peskin, Charles S

2015-01-01

In this work, we consider a stochastic oscillator described by a discrete-state continuous-time Markov chain, in which the states are arranged in a circle, and there is a constant probability per unit time of jumping from one state to the next in a specified direction around the circle. At each of a sequence of equally spaced times, the oscillator has a specified probability of being reset to a particular state. The focus of this work is the entrainment of the oscillator by this periodic but stochastic stimulus. We consider a distinguished limit, in which (i) the number of states of the oscillator approaches infinity, as does the probability per unit time of jumping from one state to the next, so that the natural mean period of the oscillator remains constant, (ii) the resetting probability approaches zero, and (iii) the period of the resetting signal approaches a multiple, by a ratio of small integers, of the natural mean period of the oscillator. In this distinguished limit, we use analytic and numerical methods to study the extent to which entrainment occurs.

Phase locking route behind complex periodic windows in a forced oscillator

NASA Astrophysics Data System (ADS)

Jan, Hengtai; Tsai, Kuo-Ting; Kuo, Li-wei

2013-09-01

Chaotic systems have complex reactions against an external driving force; even in cases with low-dimension oscillators, the routes to synchronization are diverse. We proposed a stroboscope-based method for analyzing driven chaotic systems in their phase space. According to two statistic quantities generated from time series, we could realize the system state and the driving behavior simultaneously. We demonstrated our method in a driven bi-stable system, which showed complex period windows under a proper driving force. With increasing periodic driving force, a route from interior periodic oscillation to phase synchronization through the chaos state could be found. Periodic windows could also be identified and the circumstances under which they occurred distinguished. Statistical results were supported by conditional Lyapunov exponent analysis to show the power in analyzing the unknown time series.

Integrate-and-fire models with an almost periodic input function

NASA Astrophysics Data System (ADS)

Kasprzak, Piotr; Nawrocki, Adam; Signerska-Rynkowska, Justyna

2018-02-01

We investigate leaky integrate-and-fire models (LIF models for short) driven by Stepanov and μ-almost periodic functions. Special attention is paid to the properties of the firing map and its displacement, which give information about the spiking behavior of the considered system. We provide conditions under which such maps are well-defined and are uniformly continuous. We show that the LIF models with Stepanov almost periodic inputs have uniformly almost periodic displacements. We also show that in the case of μ-almost periodic drives it may happen that the displacement map is uniformly continuous, but is not μ-almost periodic (and thus cannot be Stepanov or uniformly almost periodic). By allowing discontinuous inputs, we extend some previous results, showing, for example, that the firing rate for the LIF models with Stepanov almost periodic input exists and is unique. This is a starting point for the investigation of the dynamics of almost-periodically driven integrate-and-fire systems.

Technical manual for basic version of the Markov chain nest productivity model (MCnest)

EPA Science Inventory

The Markov Chain Nest Productivity Model (or MCnest) integrates existing toxicity information from three standardized avian toxicity tests with information on species life history and the timing of pesticide applications relative to the timing of avian breeding seasons to quantit...

User’s manual for basic version of MCnest Markov chain nest productivity model

EPA Science Inventory

The Markov Chain Nest Productivity Model (or MCnest) integrates existing toxicity information from three standardized avian toxicity tests with information on species life history and the timing of pesticide applications relative to the timing of avian breeding seasons to quantit...

A simplified parsimonious higher order multivariate Markov chain model with new convergence condition

NASA Astrophysics Data System (ADS)

Wang, Chao; Yang, Chuan-sheng

2017-09-01

In this paper, we present a simplified parsimonious higher-order multivariate Markov chain model with new convergence condition. (TPHOMMCM-NCC). Moreover, estimation method of the parameters in TPHOMMCM-NCC is give. Numerical experiments illustrate the effectiveness of TPHOMMCM-NCC.

Envelopes of Sets of Measures, Tightness, and Markov Control Processes

DOE Office of Scientific and Technical Information (OSTI.GOV)

Gonzalez-Hernandez, J.; Hernandez-Lerma, O.

1999-11-15

We introduce upper and lower envelopes for sets of measures on an arbitrary topological space, which are then used to give a tightness criterion. These concepts are applied to show the existence of optimal policies for a class of Markov control processes.

Image segmentation using hidden Markov Gauss mixture models.

PubMed

Pyun, Kyungsuk; Lim, Johan; Won, Chee Sun; Gray, Robert M

2007-07-01

Image segmentation is an important tool in image processing and can serve as an efficient front end to sophisticated algorithms and thereby simplify subsequent processing. We develop a multiclass image segmentation method using hidden Markov Gauss mixture models (HMGMMs) and provide examples of segmentation of aerial images and textures. HMGMMs incorporate supervised learning, fitting the observation probability distribution given each class by a Gauss mixture estimated using vector quantization with a minimum discrimination information (MDI) distortion. We formulate the image segmentation problem using a maximum a posteriori criteria and find the hidden states that maximize the posterior density given the observation. We estimate both the hidden Markov parameter and hidden states using a stochastic expectation-maximization algorithm. Our results demonstrate that HMGMM provides better classification in terms of Bayes risk and spatial homogeneity of the classified objects than do several popular methods, including classification and regression trees, learning vector quantization, causal hidden Markov models (HMMs), and multiresolution HMMs. The computational load of HMGMM is similar to that of the causal HMM.

Time-patterns of antibiotic exposure in poultry production--a Markov chains exploratory study of nature and consequences.

PubMed

Chauvin, C; Clement, C; Bruneau, M; Pommeret, D

2007-07-16

This article describes the use of Markov chains to explore the time-patterns of antimicrobial exposure in broiler poultry. The transition in antimicrobial exposure status (exposed/not exposed to an antimicrobial, with a distinction between exposures to the different antimicrobial classes) in extensive data collected in broiler chicken flocks from November 2003 onwards, was investigated. All Markov chains were first-order chains. Mortality rate, geographical location and slaughter semester were sources of heterogeneity between transition matrices. Transitions towards a 'no antimicrobial' exposure state were highly predominant, whatever the initial state. From a 'no antimicrobial' exposure state, the transition to beta-lactams was predominant among transitions to an antimicrobial exposure state. Transitions between antimicrobial classes were rare and variable. Switches between antimicrobial classes and repeats of a particular class were both observed. Application of Markov chains analysis to the database of the nation-wide antimicrobial resistance monitoring programme pointed out that transition probabilities between antimicrobial exposure states increased with the number of resistances in Escherichia coli strains.

Surgical gesture segmentation and recognition.

PubMed

Tao, Lingling; Zappella, Luca; Hager, Gregory D; Vidal, René

2013-01-01

Automatic surgical gesture segmentation and recognition can provide useful feedback for surgical training in robotic surgery. Most prior work in this field relies on the robot's kinematic data. Although recent work [1,2] shows that the robot's video data can be equally effective for surgical gesture recognition, the segmentation of the video into gestures is assumed to be known. In this paper, we propose a framework for joint segmentation and recognition of surgical gestures from kinematic and video data. Unlike prior work that relies on either frame-level kinematic cues, or segment-level kinematic or video cues, our approach exploits both cues by using a combined Markov/semi-Markov conditional random field (MsM-CRF) model. Our experiments show that the proposed model improves over a Markov or semi-Markov CRF when using video data alone, gives results that are comparable to state-of-the-art methods on kinematic data alone, and improves over state-of-the-art methods when combining kinematic and video data.

Efficient Learning of Continuous-Time Hidden Markov Models for Disease Progression

PubMed Central

Liu, Yu-Ying; Li, Shuang; Li, Fuxin; Song, Le; Rehg, James M.

2016-01-01

The Continuous-Time Hidden Markov Model (CT-HMM) is an attractive approach to modeling disease progression due to its ability to describe noisy observations arriving irregularly in time. However, the lack of an efficient parameter learning algorithm for CT-HMM restricts its use to very small models or requires unrealistic constraints on the state transitions. In this paper, we present the first complete characterization of efficient EM-based learning methods for CT-HMM models. We demonstrate that the learning problem consists of two challenges: the estimation of posterior state probabilities and the computation of end-state conditioned statistics. We solve the first challenge by reformulating the estimation problem in terms of an equivalent discrete time-inhomogeneous hidden Markov model. The second challenge is addressed by adapting three approaches from the continuous time Markov chain literature to the CT-HMM domain. We demonstrate the use of CT-HMMs with more than 100 states to visualize and predict disease progression using a glaucoma dataset and an Alzheimer’s disease dataset. PMID:27019571

Quantum Markov chains

NASA Astrophysics Data System (ADS)

Gudder, Stanley

2008-07-01

A new approach to quantum Markov chains is presented. We first define a transition operation matrix (TOM) as a matrix whose entries are completely positive maps whose column sums form a quantum operation. A quantum Markov chain is defined to be a pair (G,E) where G is a directed graph and E =[Eij] is a TOM whose entry Eij labels the edge from vertex j to vertex i. We think of the vertices of G as sites that a quantum system can occupy and Eij is the transition operation from site j to site i in one time step. The discrete dynamics of the system is obtained by iterating the TOM E. We next consider a special type of TOM called a transition effect matrix. In this case, there are two types of dynamics, a state dynamics and an operator dynamics. Although these two types are not identical, they are statistically equivalent. We next give examples that illustrate various properties of quantum Markov chains. We conclude by showing that our formalism generalizes the usual framework for quantum random walks.

Linear system identification via backward-time observer models

NASA Technical Reports Server (NTRS)

Juang, Jer-Nan; Phan, Minh Q.

1992-01-01

Presented here is an algorithm to compute the Markov parameters of a backward-time observer for a backward-time model from experimental input and output data. The backward-time observer Markov parameters are decomposed to obtain the backward-time system Markov parameters (backward-time pulse response samples) for the backward-time system identification. The identified backward-time system Markov parameters are used in the Eigensystem Realization Algorithm to identify a backward-time state-space model, which can be easily converted to the usual forward-time representation. If one reverses time in the model to be identified, what were damped true system modes become modes with negative damping, growing as the reversed time increases. On the other hand, the noise modes in the identification still maintain the property that they are stable. The shift from positive damping to negative damping of the true system modes allows one to distinguish these modes from noise modes. Experimental results are given to illustrate when and to what extent this concept works.

Effect of Clustering Algorithm on Establishing Markov State Model for Molecular Dynamics Simulations.

PubMed

Li, Yan; Dong, Zigang

2016-06-27

Recently, the Markov state model has been applied for kinetic analysis of molecular dynamics simulations. However, discretization of the conformational space remains a primary challenge in model building, and it is not clear how the space decomposition by distinct clustering strategies exerts influence on the model output. In this work, different clustering algorithms are employed to partition the conformational space sampled in opening and closing of fatty acid binding protein 4 as well as inactivation and activation of the epidermal growth factor receptor. Various classifications are achieved, and Markov models are set up accordingly. On the basis of the models, the total net flux and transition rate are calculated between two distinct states. Our results indicate that geometric and kinetic clustering perform equally well. The construction and outcome of Markov models are heavily dependent on the data traits. Compared to other methods, a combination of Bayesian and hierarchical clustering is feasible in identification of metastable states.

Cost-effectiveness of treating wet age-related macular degeneration at the Kuopio University Hospital in Finland based on a two-eye Markov transition model.

PubMed

Vottonen, Pasi; Kankaanpää, Eila

2016-11-01

Wet age-related macular degeneration (AMD) is the leading cause of blindness worldwide, which can be treated with regular intraocular anti-vascular endothelial growth factor (VEGF) injections. In this study, we wanted to evaluate whether less frequent injections of aflibercept would make it more cost-effective when compared with ranibizumab and low priced bevacizumab. We used a two-eye model to simulate the progression and the treatment of the disease. We selected an 8-year period, 3-month cycles and five health states based on the visual acuity of the better-seeing eye. The transition probabilities and utilities attached to the health states were gathered from previous studies. We conducted the analysis from the hospital perspective and we used the health care costs obtained from Kuopio University Hospital. The costs of intraocular adverse events were taken into account. The incremental cost-effectiveness ratio (ICER) with 3% discount rate (€/QALY) for aflibercept compared with monthly bevacizumab was 1 801 228 and when compared with ranibizumab given as needed, the ICER was minus 3 716 943. The sensitivity analysis showed that a change of 20% of the estimated model parameters or a longer follow-up period did not influence these conclusions. A two-eye Markov transition model was developed to analyse the cost-effectiveness of wet AMD treatment, as quality of life years (QALYs) are largely based on the visual acuity of the better-seeing eye. Monthly injected bevacizumab was the most cost-effective treatment and monthly ranibizumab the least effective. © 2016 Acta Ophthalmologica Scandinavica Foundation. Published by John Wiley & Sons Ltd.

Hidden Markov models reveal complexity in the diving behaviour of short-finned pilot whales

PubMed Central

Quick, Nicola J.; Isojunno, Saana; Sadykova, Dina; Bowers, Matthew; Nowacek, Douglas P.; Read, Andrew J.

2017-01-01

Diving behaviour of short-finned pilot whales is often described by two states; deep foraging and shallow, non-foraging dives. However, this simple classification system ignores much of the variation that occurs during subsurface periods. We used multi-state hidden Markov models (HMM) to characterize states of diving behaviour and the transitions between states in short-finned pilot whales. We used three parameters (number of buzzes, maximum dive depth and duration) measured in 259 dives by digital acoustic recording tags (DTAGs) deployed on 20 individual whales off Cape Hatteras, North Carolina, USA. The HMM identified a four-state model as the best descriptor of diving behaviour. The state-dependent distributions for the diving parameters showed variation between states, indicative of different diving behaviours. Transition probabilities were considerably higher for state persistence than state switching, indicating that dive types occurred in bouts. Our results indicate that subsurface behaviour in short-finned pilot whales is more complex than a simple dichotomy of deep and shallow diving states, and labelling all subsurface behaviour as deep dives or shallow dives discounts a significant amount of important variation. We discuss potential drivers of these patterns, including variation in foraging success, prey availability and selection, bathymetry, physiological constraints and socially mediated behaviour. PMID:28361954

Service-Oriented Node Scheduling Scheme for Wireless Sensor Networks Using Markov Random Field Model

PubMed Central

Cheng, Hongju; Su, Zhihuang; Lloret, Jaime; Chen, Guolong

2014-01-01

Future wireless sensor networks are expected to provide various sensing services and energy efficiency is one of the most important criterions. The node scheduling strategy aims to increase network lifetime by selecting a set of sensor nodes to provide the required sensing services in a periodic manner. In this paper, we are concerned with the service-oriented node scheduling problem to provide multiple sensing services while maximizing the network lifetime. We firstly introduce how to model the data correlation for different services by using Markov Random Field (MRF) model. Secondly, we formulate the service-oriented node scheduling issue into three different problems, namely, the multi-service data denoising problem which aims at minimizing the noise level of sensed data, the representative node selection problem concerning with selecting a number of active nodes while determining the services they provide, and the multi-service node scheduling problem which aims at maximizing the network lifetime. Thirdly, we propose a Multi-service Data Denoising (MDD) algorithm, a novel multi-service Representative node Selection and service Determination (RSD) algorithm, and a novel MRF-based Multi-service Node Scheduling (MMNS) scheme to solve the above three problems respectively. Finally, extensive experiments demonstrate that the proposed scheme efficiently extends the network lifetime. PMID:25384005

Understanding the ontogeny of foraging behaviour: insights from combining marine predator bio-logging with satellite-derived oceanography in hidden Markov models.

PubMed

Grecian, W James; Lane, Jude V; Michelot, Théo; Wade, Helen M; Hamer, Keith C

2018-06-01

The development of foraging strategies that enable juveniles to efficiently identify and exploit predictable habitat features is critical for survival and long-term fitness. In the marine environment, meso- and sub-mesoscale features such as oceanographic fronts offer a visible cue to enhanced foraging conditions, but how individuals learn to identify these features is a mystery. In this study, we investigate age-related differences in the fine-scale foraging behaviour of adult (aged ≥ 5 years) and immature (aged 2-4 years) northern gannets Morus bassanus Using high-resolution GPS-loggers, we reveal that adults have a much narrower foraging distribution than immature birds and much higher individual foraging site fidelity. By conditioning the transition probabilities of a hidden Markov model on satellite-derived measures of frontal activity, we then demonstrate that adults show a stronger response to frontal activity than immature birds, and are more likely to commence foraging behaviour as frontal intensity increases. Together, these results indicate that adult gannets are more proficient foragers than immatures, supporting the hypothesis that foraging specializations are learned during individual exploratory behaviour in early life. Such memory-based individual foraging strategies may also explain the extended period of immaturity observed in gannets and many other long-lived species. © 2018 The Authors.

Effect of Parametric Dichotomic Markov Noise on the Properties of Chaotic Transitions in Dynamical Systems

NASA Astrophysics Data System (ADS)

Gac, J. M.; Żebrowski, J. J.

A chaotic transition occurs when a continuous change of one of the parameters of the system causes a discontinuous change in the properties of the chaotic attractor of the system. Such phenomena are present in many dynamical systems, in which a chaotic behavior occurs. The best known of these transitions are: the period-doubling bifurcation cascade, intermittency and crises. The effect of dichotomous Markov noise (DMN) on the properties of systems with chaotic transitions is discussed. DMN is a very simple two-valued stochastic process, with constant transition rates between the two states. In spite of its simplicity, this kind of noise is a very powerful tool to describe various phenomena present in many physical, chemical or biological systems. Many interesting phenomena induced by DMN are known. However, there is no research on the effect of this kind of noise on intermittency or crises. We present the change of the mean laminar phase length and of laminar phase length distribution caused by DMN modulating the parameters of a system with intermittency and the modification of the mean life time on the pre-crisis attractor in the case of a boundary crisis. The results obtained analytically are compared with numerical simulations for several simple dynamical systems.

Counter-rotating effects and entanglement dynamics in strongly coupled quantum-emitter-metallic-nanoparticle structures

NASA Astrophysics Data System (ADS)

Iliopoulos, Nikos; Thanopulos, Ioannis; Yannopapas, Vassilios; Paspalakis, Emmanuel

2018-03-01

We study the spontaneous emission of a two-level quantum emitter next to a plasmonic nanoparticle beyond the Markovian approximation and the rotating-wave approximation (RWA) by combining quantum dynamics and classical electromagnetic calculations. For emitters with decay times in the picosecond to nanosecond time regime, as well as located at distances from the nanoparticle up to its radius, the dynamics with and without the RWA and the transition from the non-Markovian to the Markovian regime are investigated. For emitters with longer decay times, the Markov approximation proves to be adequate for distances larger than half the nanoparticle radius. However, the RWA is correct for all distances of the emitter from the nanoparticle. For short decay time emitters, the Markov approximation and RWA are both inadequate, with only the RWA becoming valid again at a distance larger than half the nanoparticle radius. We also show that the entanglement dynamics of two initially entangled qubits interacting independently with the nanoparticle may have a strong non-Markovian character when counter-rotating effects are included. Interesting effects such as entanglement sudden death, periodic entanglement revival, entanglement oscillations, and entanglement trapping are further observed when different initial two-qubit states and different distances between the qubit and the nanoparticle are considered.

Interaction quenched ultracold few-boson ensembles in periodically driven lattices

NASA Astrophysics Data System (ADS)

Mistakidis, Simeon; Schmelcher, Peter; Theory Group of Fundamental Processes in Quantum Physics Team

2017-04-01

The out-of-equilibrium dynamics of interaction quenched finite ultracold bosonic ensembles in periodically driven one-dimensional optical lattices is investigated. It is shown that periodic driving enforces the bosons in the outer wells of the finite lattice to exhibit out-of-phase dipole-like modes, while in the central well the atomic cloud experiences a local breathing mode. The dynamical behavior is investigated with varying driving frequency, revealing a resonant-like behavior of the intra-well dynamics. An interaction quench in the periodically driven lattice gives rise to admixtures of different excitations in the outer wells, an enhanced breathing in the center and an amplification of the tunneling dynamics. We observe then multiple resonances between the inter- and intra-well dynamics at different quench amplitudes, with the position of the resonances being tunable via the driving frequency. Our results pave the way for future investigations on the use of combined driving protocols in order to excite different inter- and intra-well modes and to subsequently control them. Deutsche Forschungsgemeinschaft (DFG) in the framework of the SFB 925 ``Light induced dynamics and control of correlated quantum systems''.

Nonparametric model validations for hidden Markov models with applications in financial econometrics

PubMed Central

Zhao, Zhibiao

2011-01-01

We address the nonparametric model validation problem for hidden Markov models with partially observable variables and hidden states. We achieve this goal by constructing a nonparametric simultaneous confidence envelope for transition density function of the observable variables and checking whether the parametric density estimate is contained within such an envelope. Our specification test procedure is motivated by a functional connection between the transition density of the observable variables and the Markov transition kernel of the hidden states. Our approach is applicable for continuous time diffusion models, stochastic volatility models, nonlinear time series models, and models with market microstructure noise. PMID:21750601

Stochastic-shielding approximation of Markov chains and its application to efficiently simulate random ion-channel gating.

PubMed

Schmandt, Nicolaus T; Galán, Roberto F

2012-09-14

Markov chains provide realistic models of numerous stochastic processes in nature. We demonstrate that in any Markov chain, the change in occupation number in state A is correlated to the change in occupation number in state B if and only if A and B are directly connected. This implies that if we are only interested in state A, fluctuations in B may be replaced with their mean if state B is not directly connected to A, which shortens computing time considerably. We show the accuracy and efficacy of our approximation theoretically and in simulations of stochastic ion-channel gating in neurons.

A stochastic hybrid systems based framework for modeling dependent failure processes

PubMed Central

Fan, Mengfei; Zeng, Zhiguo; Zio, Enrico; Kang, Rui; Chen, Ying

2017-01-01

In this paper, we develop a framework to model and analyze systems that are subject to dependent, competing degradation processes and random shocks. The degradation processes are described by stochastic differential equations, whereas transitions between the system discrete states are triggered by random shocks. The modeling is, then, based on Stochastic Hybrid Systems (SHS), whose state space is comprised of a continuous state determined by stochastic differential equations and a discrete state driven by stochastic transitions and reset maps. A set of differential equations are derived to characterize the conditional moments of the state variables. System reliability and its lower bounds are estimated from these conditional moments, using the First Order Second Moment (FOSM) method and Markov inequality, respectively. The developed framework is applied to model three dependent failure processes from literature and a comparison is made to Monte Carlo simulations. The results demonstrate that the developed framework is able to yield an accurate estimation of reliability with less computational costs compared to traditional Monte Carlo-based methods. PMID:28231313

A stochastic hybrid systems based framework for modeling dependent failure processes.

PubMed

Fan, Mengfei; Zeng, Zhiguo; Zio, Enrico; Kang, Rui; Chen, Ying

2017-01-01

In this paper, we develop a framework to model and analyze systems that are subject to dependent, competing degradation processes and random shocks. The degradation processes are described by stochastic differential equations, whereas transitions between the system discrete states are triggered by random shocks. The modeling is, then, based on Stochastic Hybrid Systems (SHS), whose state space is comprised of a continuous state determined by stochastic differential equations and a discrete state driven by stochastic transitions and reset maps. A set of differential equations are derived to characterize the conditional moments of the state variables. System reliability and its lower bounds are estimated from these conditional moments, using the First Order Second Moment (FOSM) method and Markov inequality, respectively. The developed framework is applied to model three dependent failure processes from literature and a comparison is made to Monte Carlo simulations. The results demonstrate that the developed framework is able to yield an accurate estimation of reliability with less computational costs compared to traditional Monte Carlo-based methods.

Revisiting CMB constraints on warm inflation

NASA Astrophysics Data System (ADS)

Arya, Richa; Dasgupta, Arnab; Goswami, Gaurav; Prasad, Jayanti; Rangarajan, Raghavan

2018-02-01

We revisit the constraints that Planck 2015 temperature, polarization and lensing data impose on the parameters of warm inflation. To this end, we study warm inflation driven by a single scalar field with a quartic self interaction potential in the weak dissipative regime. We analyse the effect of the parameters of warm inflation, namely, the inflaton self coupling λ and the inflaton dissipation parameter QP on the CMB angular power spectrum. We constrain λ and QP for 50 and 60 number of e-foldings with the full Planck 2015 data (TT, TE, EE + lowP and lensing) by performing a Markov-Chain Monte Carlo analysis using the publicly available code CosmoMC and obtain the joint as well as marginalized distributions of those parameters. We present our results in the form of mean and 68 % confidence limits on the parameters and also highlight the degeneracy between λ and QP in our analysis. From this analysis we show how warm inflation parameters can be well constrained using the Planck 2015 data.

Continuously growing rodent molars result from a predictable quantitative evolutionary change over 50 million years

PubMed Central

Mushegyan, Vagan; Eronen, Jussi T.; Lawing, A. Michelle; Sharir, Amnon; Janis, Christine; Jernvall, Jukka; Klein, Ophir D.

2015-01-01

Summary The fossil record is widely informative about evolution, but fossils are not systematically used to study the evolution of stem cell-driven renewal. Here, we examined evolution of the continuous growth (hypselodonty) of rodent molar teeth, which is fuelled by the presence of dental stem cells. We studied occurrences of 3500 North American rodent fossils, ranging from 50 million years ago (mya) to 2 mya. We examined changes in molar height to determine if evolution of hypselodonty shows distinct patterns in the fossil record, and we found that hypselodont taxa emerged through intermediate forms of increasing crown height. Next, we designed a Markov simulation model, which replicated molar height increases throughout the Cenozoic, and, moreover, evolution of hypselodonty. Thus, by extension, the retention of the adult stem-cell niche appears to be a predictable quantitative rather than a stochastic qualitative process. Our analyses predict that hypselodonty will eventually become the dominant phenotype. PMID:25921530

Assessing Model Fitting of Megamaser Disks with Simulated Observations

NASA Astrophysics Data System (ADS)

Han, Jiwon; Braatz, James; Pesce, Dominic

2018-01-01

The Megamaser Cosmology Project (MCP) measures the Hubble Constant by determining distances to galaxies with observations of 22 GHz H20 megamasers. The megamasers arise in the circumnuclear accretion disks of active galaxies. In this research, we aim to improve the estimation of systematic errors in MCP measurements. Currently, the MCP fits a disk model to the observed maser data with a Markov Chain Monte Carlo (MCMC) code. The disk model is described by up to 14 global parameters, including up to 6 that describe the disk warping. We first assess the model by generating synthetic datasets in which the locations and dynamics of the maser spots are exactly known, and fitting the model to these data. By doing so, we can also test the effects of unmodeled substructure on the estimated uncertainties. Furthermore, in order to gain better understanding of the physics behind accretion disk warping, we develop a physics-driven model for the warp and test it with the MCMC approach.

A new genome-mining tool redefines the lasso peptide biosynthetic landscape

PubMed Central

Tietz, Jonathan I.; Schwalen, Christopher J.; Patel, Parth S.; Maxson, Tucker; Blair, Patricia M.; Tai, Hua-Chia; Zakai, Uzma I.; Mitchell, Douglas A.

2016-01-01

Ribosomally synthesized and post-translationally modified peptide (RiPP) natural products are attractive for genome-driven discovery and re-engineering, but limitations in bioinformatic methods and exponentially increasing genomic data make large-scale mining difficult. We report RODEO (Rapid ORF Description and Evaluation Online), which combines hidden Markov model-based analysis, heuristic scoring, and machine learning to identify biosynthetic gene clusters and predict RiPP precursor peptides. We initially focused on lasso peptides, which display intriguing physiochemical properties and bioactivities, but their hypervariability renders them challenging prospects for automated mining. Our approach yielded the most comprehensive mapping of lasso peptide space, revealing >1,300 compounds. We characterized the structures and bioactivities of six lasso peptides, prioritized based on predicted structural novelty, including an unprecedented handcuff-like topology and another with a citrulline modification exceptionally rare among bacteria. These combined insights significantly expand the knowledge of lasso peptides, and more broadly, provide a framework for future genome-mining efforts. PMID:28244986

Integrable dissipative exclusion process: Correlation functions and physical properties

NASA Astrophysics Data System (ADS)

Crampe, N.; Ragoucy, E.; Rittenberg, V.; Vanicat, M.

2016-09-01

We study a one-parameter generalization of the symmetric simple exclusion process on a one-dimensional lattice. In addition to the usual dynamics (where particles can hop with equal rates to the left or to the right with an exclusion constraint), annihilation and creation of pairs can occur. The system is driven out of equilibrium by two reservoirs at the boundaries. In this setting the model is still integrable: it is related to the open XXZ spin chain through a gauge transformation. This allows us to compute the full spectrum of the Markov matrix using Bethe equations. We also show that the stationary state can be expressed in a matrix product form permitting to compute the multipoints correlation functions as well as the mean value of the lattice and the creation-annihilation currents. Finally, the variance of the lattice current is computed for a finite-size system. In the thermodynamic limit, it matches the value obtained from the associated macroscopic fluctuation theory.

Hideen Markov Models and Neural Networks for Fault Detection in Dynamic Systems

NASA Technical Reports Server (NTRS)

Smyth, Padhraic

1994-01-01

None given. (From conclusion): Neural networks plus Hidden Markov Models(HMM)can provide excellene detection and false alarm rate performance in fault detection applications. Modified models allow for novelty detection. Also covers some key contributions of neural network model, and application status.

User's Manual MCnest - Markov Chain Nest Productivity Model Version 2.0

EPA Science Inventory

The Markov chain nest productivity model, or MCnest, is a set of algorithms for integrating the results of avian toxicity tests with reproductive life-history data to project the relative magnitude of chemical effects on avian reproduction. The mathematical foundation of MCnest i...

Document Ranking Based upon Markov Chains.

ERIC Educational Resources Information Center

Danilowicz, Czeslaw; Balinski, Jaroslaw

2001-01-01

Considers how the order of documents in information retrieval responses are determined and introduces a method that uses a probabilistic model of a document set where documents are regarded as states of a Markov chain and where transition probabilities are directly proportional to similarities between documents. (Author/LRW)

Using Markov Chain Analyses in Counselor Education Research

ERIC Educational Resources Information Center

Duys, David K.; Headrick, Todd C.

2004-01-01

This study examined the efficacy of an infrequently used statistical analysis in counselor education research. A Markov chain analysis was used to examine hypothesized differences between students' use of counseling skills in an introductory course. Thirty graduate students participated in the study. Independent raters identified the microskills…

Quantum Markov Semigroups with Unbounded Generator and Time Evolution of the Support Projection of a State

NASA Astrophysics Data System (ADS)

Gliouez, Souhir; Hachicha, Skander; Nasroui, Ikbel

We characterize the support projection of a state evolving under the action of a quantum Markov semigroup with unbounded generator represented in the generalized GKSL form and a quantum version of the classical Lévy-Austin-Ornstein theorem.

Evidence of strategic periodicities in collective conflict dynamics.

PubMed

Dedeo, Simon; Krakauer, David; Flack, Jessica

2011-09-07

We analyse the timescales of conflict decision-making in a primate society. We present evidence for multiple, periodic timescales associated with social decision-making and behavioural patterns. We demonstrate the existence of periodicities that are not directly coupled to environmental cycles or known ultraridian mechanisms. Among specific biological and socially defined demographic classes, periodicities span timescales between hours and days. Our results indicate that these periodicities are not driven by exogenous or internal regularities but are instead driven by strategic responses to social interaction patterns. Analyses also reveal that a class of individuals, playing a critical functional role, policing, have a signature timescale of the order of 1 h. We propose a classification of behavioural timescales analogous to those of the nervous system, with high frequency, or α-scale, behaviour occurring on hour-long scales, through to multi-hour, or β-scale, behaviour, and, finally γ periodicities observed on a timescale of days.

Surgical motion characterization in simulated needle insertion procedures

NASA Astrophysics Data System (ADS)

Holden, Matthew S.; Ungi, Tamas; Sargent, Derek; McGraw, Robert C.; Fichtinger, Gabor

2012-02-01

PURPOSE: Evaluation of surgical performance in image-guided needle insertions is of emerging interest, to both promote patient safety and improve the efficiency and effectiveness of training. The purpose of this study was to determine if a Markov model-based algorithm can more accurately segment a needle-based surgical procedure into its five constituent tasks than a simple threshold-based algorithm. METHODS: Simulated needle trajectories were generated with known ground truth segmentation by a synthetic procedural data generator, with random noise added to each degree of freedom of motion. The respective learning algorithms were trained, and then tested on different procedures to determine task segmentation accuracy. In the threshold-based algorithm, a change in tasks was detected when the needle crossed a position/velocity threshold. In the Markov model-based algorithm, task segmentation was performed by identifying the sequence of Markov models most likely to have produced the series of observations. RESULTS: For amplitudes of translational noise greater than 0.01mm, the Markov model-based algorithm was significantly more accurate in task segmentation than the threshold-based algorithm (82.3% vs. 49.9%, p<0.001 for amplitude 10.0mm). For amplitudes less than 0.01mm, the two algorithms produced insignificantly different results. CONCLUSION: Task segmentation of simulated needle insertion procedures was improved by using a Markov model-based algorithm as opposed to a threshold-based algorithm for procedures involving translational noise.

Stochastic modelling of a single ion channel: an alternating renewal approach with application to limited time resolution.

PubMed

Milne, R K; Yeo, G F; Edeson, R O; Madsen, B W

1988-04-22

Stochastic models of ion channels have been based largely on Markov theory where individual states and transition rates must be specified, and sojourn-time densities for each state are constrained to be exponential. This study presents an approach based on random-sum methods and alternating-renewal theory, allowing individual states to be grouped into classes provided the successive sojourn times in a given class are independent and identically distributed. Under these conditions Markov models form a special case. The utility of the approach is illustrated by considering the effects of limited time resolution (modelled by using a discrete detection limit, xi) on the properties of observable events, with emphasis on the observed open-time (xi-open-time). The cumulants and Laplace transform for a xi-open-time are derived for a range of Markov and non-Markov models; several useful approximations to the xi-open-time density function are presented. Numerical studies show that the effects of limited time resolution can be extreme, and also highlight the relative importance of the various model parameters. The theory could form a basis for future inferential studies in which parameter estimation takes account of limited time resolution in single channel records. Appendixes include relevant results concerning random sums and a discussion of the role of exponential distributions in Markov models.

Nonequilibrium dynamic phases in driven vortex lattices with periodic pinning

NASA Astrophysics Data System (ADS)

Reichhardt, Charles Michael

1998-12-01

We present the results of an extensive series of simulations of flux-gradient and current driven vortices interacting with either random or periodically arranged pinning sites. First, we consider flux-gradient-driven simulations of superconducting vortices interacting with strong randomly-distributed columnar pinning defects, as an external field H(t) is quasi-statically swept from zero through a matching field Bsb{phi}. Here, we find significant changes in the behavior of the local flux density B(x, y, H(t)), magnetization M(H(t)), critical current Jsb{c}(B(t)), and the individual vortex flow paths, as the local flux density crosses Bsb{phi}. Further, we find that for a given pin density, Jsb{c}(B) can be enhanced by maximizing the distance between the pins for B < Bsb{phi}. For the case of periodic pinning sites as a function of applied field, we find a rich variety of ordered and partially-ordered vortex lattice configurations. We present formulas that predict the matching fields at which commensurate vortex configurations occur and the vortex lattice orientation with respect to the pinning lattice. Our results are in excellent agreement with recent imaging experiments on square pinning arrays (K. Harada et al., Science 274, 1167 (1996)). For current driven simulations with periodic pinning we find a remarkable number of dynamical plastic flow phases. Signatures of the transitions between these different dynamical phases include sudden jumps in the current-voltage curves, hysteresis, as well as marked changes in the vortex trajectories and vortex lattice order. These phases are outlined in a series of dynamic phase diagrams. We show that several of these phases and their phase-boundaries can be understood in terms of analytical arguments. Finally, when the vortex lattice is driven at varying angles with respect to the underlying periodic pinning array, the transverse voltage-current V(I) curves show a series of mode-locked plateaus with the overall V(I) forming a devil's staircase structure.

Markov Networks of Collateral Resistance: National Antimicrobial Resistance Monitoring System Surveillance Results from Escherichia coli Isolates, 2004-2012

PubMed Central

Love, William J.; Zawack, Kelson A.; Booth, James G.; Grӧhn, Yrjo T.

2016-01-01

Surveillance of antimicrobial resistance (AMR) is an important component of public health. Antimicrobial drug use generates selective pressure that may lead to resistance against to the administered drug, and may also select for collateral resistances to other drugs. Analysis of AMR surveillance data has focused on resistance to individual drugs but joint distributions of resistance in bacterial populations are infrequently analyzed and reported. New methods are needed to characterize and communicate joint resistance distributions. Markov networks are a class of graphical models that define connections, or edges, between pairs of variables with non-zero partial correlations and are used here to describe AMR resistance relationships. The graphical least absolute shrinkage and selection operator is used to estimate sparse Markov networks from AMR surveillance data. The method is demonstrated using a subset of Escherichia coli isolates collected by the National Antimicrobial Resistance Monitoring System between 2004 and 2012 which included AMR results for 16 drugs from 14418 isolates. Of the 119 possible unique edges, 33 unique edges were identified at least once during the study period and graphical density ranged from 16.2% to 24.8%. Two frequent dense subgraphs were noted, one containing the five β-lactam drugs and the other containing both sulfonamides, three aminoglycosides, and tetracycline. Density did not appear to change over time (p = 0.71). Unweighted modularity did not appear to change over time (p = 0.18), but a significant decreasing trend was noted in the modularity of the weighted networks (p < 0.005) indicating relationships between drugs of different classes tended to increase in strength and frequency over time compared to relationships between drugs of the same class. The current method provides a novel method to study the joint resistance distribution, but additional work is required to unite the underlying biological and genetic characteristics of the isolates with the current results derived from phenotypic data. PMID:27851767

Markov Networks of Collateral Resistance: National Antimicrobial Resistance Monitoring System Surveillance Results from Escherichia coli Isolates, 2004-2012.

PubMed

Love, William J; Zawack, Kelson A; Booth, James G; Grӧhn, Yrjo T; Lanzas, Cristina

2016-11-01

Surveillance of antimicrobial resistance (AMR) is an important component of public health. Antimicrobial drug use generates selective pressure that may lead to resistance against to the administered drug, and may also select for collateral resistances to other drugs. Analysis of AMR surveillance data has focused on resistance to individual drugs but joint distributions of resistance in bacterial populations are infrequently analyzed and reported. New methods are needed to characterize and communicate joint resistance distributions. Markov networks are a class of graphical models that define connections, or edges, between pairs of variables with non-zero partial correlations and are used here to describe AMR resistance relationships. The graphical least absolute shrinkage and selection operator is used to estimate sparse Markov networks from AMR surveillance data. The method is demonstrated using a subset of Escherichia coli isolates collected by the National Antimicrobial Resistance Monitoring System between 2004 and 2012 which included AMR results for 16 drugs from 14418 isolates. Of the 119 possible unique edges, 33 unique edges were identified at least once during the study period and graphical density ranged from 16.2% to 24.8%. Two frequent dense subgraphs were noted, one containing the five β-lactam drugs and the other containing both sulfonamides, three aminoglycosides, and tetracycline. Density did not appear to change over time (p = 0.71). Unweighted modularity did not appear to change over time (p = 0.18), but a significant decreasing trend was noted in the modularity of the weighted networks (p < 0.005) indicating relationships between drugs of different classes tended to increase in strength and frequency over time compared to relationships between drugs of the same class. The current method provides a novel method to study the joint resistance distribution, but additional work is required to unite the underlying biological and genetic characteristics of the isolates with the current results derived from phenotypic data.

Understanding Past Population Dynamics: Bayesian Coalescent-Based Modeling with Covariates

PubMed Central

Gill, Mandev S.; Lemey, Philippe; Bennett, Shannon N.; Biek, Roman; Suchard, Marc A.

2016-01-01

Effective population size characterizes the genetic variability in a population and is a parameter of paramount importance in population genetics and evolutionary biology. Kingman’s coalescent process enables inference of past population dynamics directly from molecular sequence data, and researchers have developed a number of flexible coalescent-based models for Bayesian nonparametric estimation of the effective population size as a function of time. Major goals of demographic reconstruction include identifying driving factors of effective population size, and understanding the association between the effective population size and such factors. Building upon Bayesian nonparametric coalescent-based approaches, we introduce a flexible framework that incorporates time-varying covariates that exploit Gaussian Markov random fields to achieve temporal smoothing of effective population size trajectories. To approximate the posterior distribution, we adapt efficient Markov chain Monte Carlo algorithms designed for highly structured Gaussian models. Incorporating covariates into the demographic inference framework enables the modeling of associations between the effective population size and covariates while accounting for uncertainty in population histories. Furthermore, it can lead to more precise estimates of population dynamics. We apply our model to four examples. We reconstruct the demographic history of raccoon rabies in North America and find a significant association with the spatiotemporal spread of the outbreak. Next, we examine the effective population size trajectory of the DENV-4 virus in Puerto Rico along with viral isolate count data and find similar cyclic patterns. We compare the population history of the HIV-1 CRF02_AG clade in Cameroon with HIV incidence and prevalence data and find that the effective population size is more reflective of incidence rate. Finally, we explore the hypothesis that the population dynamics of musk ox during the Late Quaternary period were related to climate change. [Coalescent; effective population size; Gaussian Markov random fields; phylodynamics; phylogenetics; population genetics. PMID:27368344

OPTIMIZING OBSERVER EFFORT FOR FIELD DETECTION OF REPRODUCTIVE EFFECTS IN BIRDS

EPA Science Inventory

Avian nest survival is best viewed as a Markov process with two absorbing states, death and fledging. We present a column-stochastic Markov chain from which all major Mayfield formulations of daily nest-survival can be derived contingent upon the degree of observer knowledge of e...

Two Aspects of the Simplex Model: Goodness of Fit to Linear Growth Curve Structures and the Analysis of Mean Trends.

ERIC Educational Resources Information Center

Mandys, Frantisek; Dolan, Conor V.; Molenaar, Peter C. M.

1994-01-01

Studied the conditions under which the quasi-Markov simplex model fits a linear growth curve covariance structure and determined when the model is rejected. Presents a quasi-Markov simplex model with structured means and gives an example. (SLD)

Discrete Latent Markov Models for Normally Distributed Response Data

ERIC Educational Resources Information Center

Schmittmann, Verena D.; Dolan, Conor V.; van der Maas, Han L. J.; Neale, Michael C.

2005-01-01

Van de Pol and Langeheine (1990) presented a general framework for Markov modeling of repeatedly measured discrete data. We discuss analogical single indicator models for normally distributed responses. In contrast to discrete models, which have been studied extensively, analogical continuous response models have hardly been considered. These…

Avian life history profiles for use in the Markov chain nest productivity model (MCnest)

EPA Science Inventory

The Markov Chain nest productivity model, or MCnest, quantitatively estimates the effects of pesticides or other toxic chemicals on annual reproductive success of avian species (Bennett and Etterson 2013, Etterson and Bennett 2013). The Basic Version of MCnest was developed as a...

Exploring Mass Perception with Markov Chain Monte Carlo

ERIC Educational Resources Information Center

Cohen, Andrew L.; Ross, Michael G.

2009-01-01

Several previous studies have examined the ability to judge the relative mass of objects in idealized collisions. With a newly developed technique of psychological Markov chain Monte Carlo sampling (A. N. Sanborn & T. L. Griffiths, 2008), this work explores participants; perceptions of different collision mass ratios. The results reveal…

Using the Pearson Distribution for Synthesis of the Suboptimal Algorithms for Filtering Multi-Dimensional Markov Processes

NASA Astrophysics Data System (ADS)

Mit'kin, A. S.; Pogorelov, V. A.; Chub, E. G.

2015-08-01

We consider the method of constructing the suboptimal filter on the basis of approximating the a posteriori probability density of the multidimensional Markov process by the Pearson distributions. The proposed method can efficiently be used for approximating asymmetric, excessive, and finite densities.

Improving Markov Chain Models for Road Profiles Simulation via Definition of States

DTIC Science & Technology

2012-04-01

wavelet transform in pavement profile analysis," Vehicle System Dynamics: International Journal of Vehicle Mechanics and Mobility, vol. 47, no. 4...34Estimating Markov Transition Probabilities from Micro -Unit Data," Journal of the Royal Statistical Society. Series C (Applied Statistics), pp. 355-371

Density Control of Multi-Agent Systems with Safety Constraints: A Markov Chain Approach

NASA Astrophysics Data System (ADS)

Demirer, Nazli

The control of systems with autonomous mobile agents has been a point of interest recently, with many applications like surveillance, coverage, searching over an area with probabilistic target locations or exploring an area. In all of these applications, the main goal of the swarm is to distribute itself over an operational space to achieve mission objectives specified by the density of swarm. This research focuses on the problem of controlling the distribution of multi-agent systems considering a hierarchical control structure where the whole swarm coordination is achieved at the high-level and individual vehicle/agent control is managed at the low-level. High-level coordination algorithms uses macroscopic models that describes the collective behavior of the whole swarm and specify the agent motion commands, whose execution will lead to the desired swarm behavior. The low-level control laws execute the motion to follow these commands at the agent level. The main objective of this research is to develop high-level decision control policies and algorithms to achieve physically realizable commanding of the agents by imposing mission constraints on the distribution. We also make some connections with decentralized low-level motion control. This dissertation proposes a Markov chain based method to control the density distribution of the whole system where the implementation can be achieved in a decentralized manner with no communication between agents since establishing communication with large number of agents is highly challenging. The ultimate goal is to guide the overall density distribution of the system to a prescribed steady-state desired distribution while satisfying desired transition and safety constraints. Here, the desired distribution is determined based on the mission requirements, for example in the application of area search, the desired distribution should match closely with the probabilistic target locations. The proposed method is applicable for both systems with a single agent and systems with large number of agents due to the probabilistic nature, where the probability distribution of each agent's state evolves according to a finite-state and discrete-time Markov chain (MC). Hence, designing proper decision control policies requires numerically tractable solution methods for the synthesis of Markov chains. The synthesis problem has the form of a Linear Matrix Inequality Problem (LMI), with LMI formulation of the constraints. To this end, we propose convex necessary and sufficient conditions for safety constraints in Markov chains, which is a novel result in the Markov chain literature. In addition to LMI-based, offline, Markov matrix synthesis method, we also propose a QP-based, online, method to compute a time-varying Markov matrix based on the real-time density feedback. Both problems are convex optimization problems that can be solved in a reliable and tractable way, utilizing existing tools in the literature. A Low Earth Orbit (LEO) swarm simulations are presented to validate the effectiveness of the proposed algorithms. Another problem tackled as a part of this research is the generalization of the density control problem to autonomous mobile agents with two control modes: ON and OFF. Here, each mode consists of a (possibly overlapping) finite set of actions, that is, there exist a set of actions for the ON mode and another set for the OFF mode. We give formulation for a new Markov chain synthesis problem, with additional measurements for the state transitions, where a policy is designed to ensure desired safety and convergence properties for the underlying Markov chain.

Period variations of Algol-type eclipsing binaries AD And, TWCas and IV Cas

NASA Astrophysics Data System (ADS)

Parimucha, Štefan; Gajdoš, Pavol; Kudak, Viktor; Fedurco, Miroslav; Vaňko, Martin

2018-04-01

We present new analyses of variations in O – C diagrams of three Algol-type eclipsing binary stars: AD And, TW Cas and IV Cas. We have used all published minima times (including visual and photographic) as well as newly determined ones from our and SuperWasp observations. We determined orbital parameters of 3rd bodies in the systems with statistically significant errors, using our code based on genetic algorithms and Markov chain Monte Carlo simulations. We confirmed the multiple nature of AD And and the triple-star model of TW Cas, and we proposed a quadruple-star model of IV Cas.

Pattern identification in time-course gene expression data with the CoGAPS matrix factorization.

PubMed

Fertig, Elana J; Stein-O'Brien, Genevieve; Jaffe, Andrew; Colantuoni, Carlo

2014-01-01

Patterns in time-course gene expression data can represent the biological processes that are active over the measured time period. However, the orthogonality constraint in standard pattern-finding algorithms, including notably principal components analysis (PCA), confounds expression changes resulting from simultaneous, non-orthogonal biological processes. Previously, we have shown that Markov chain Monte Carlo nonnegative matrix factorization algorithms are particularly adept at distinguishing such concurrent patterns. One such matrix factorization is implemented in the software package CoGAPS. We describe the application of this software and several technical considerations for identification of age-related patterns in a public, prefrontal cortex gene expression dataset.

Corruption dynamics model

NASA Astrophysics Data System (ADS)

Malafeyev, O. A.; Nemnyugin, S. A.; Rylow, D.; Kolpak, E. P.; Awasthi, Achal

2017-07-01

The corruption dynamics is analyzed by means of the lattice model which is similar to the three-dimensional Ising model. Agents placed at nodes of the corrupt network periodically choose to perfom or not to perform the act of corruption at gain or loss while making decisions based on the process history. The gain value and its dynamics are defined by means of the Markov stochastic process modelling with parameters established in accordance with the influence of external and individual factors on the agent's gain. The model is formulated algorithmically and is studied by means of the computer simulation. Numerical results are obtained which demonstrate asymptotic behaviour of the corruption network under various conditions.

SPOTting model parameters using a ready-made Python package

NASA Astrophysics Data System (ADS)

Houska, Tobias; Kraft, Philipp; Breuer, Lutz

2015-04-01

The selection and parameterization of reliable process descriptions in ecological modelling is driven by several uncertainties. The procedure is highly dependent on various criteria, like the used algorithm, the likelihood function selected and the definition of the prior parameter distributions. A wide variety of tools have been developed in the past decades to optimize parameters. Some of the tools are closed source. Due to this, the choice for a specific parameter estimation method is sometimes more dependent on its availability than the performance. A toolbox with a large set of methods can support users in deciding about the most suitable method. Further, it enables to test and compare different methods. We developed the SPOT (Statistical Parameter Optimization Tool), an open source python package containing a comprehensive set of modules, to analyze and optimize parameters of (environmental) models. SPOT comes along with a selected set of algorithms for parameter optimization and uncertainty analyses (Monte Carlo, MC; Latin Hypercube Sampling, LHS; Maximum Likelihood, MLE; Markov Chain Monte Carlo, MCMC; Scuffled Complex Evolution, SCE-UA; Differential Evolution Markov Chain, DE-MCZ), together with several likelihood functions (Bias, (log-) Nash-Sutcliff model efficiency, Correlation Coefficient, Coefficient of Determination, Covariance, (Decomposed-, Relative-, Root-) Mean Squared Error, Mean Absolute Error, Agreement Index) and prior distributions (Binomial, Chi-Square, Dirichlet, Exponential, Laplace, (log-, multivariate-) Normal, Pareto, Poisson, Cauchy, Uniform, Weibull) to sample from. The model-independent structure makes it suitable to analyze a wide range of applications. We apply all algorithms of the SPOT package in three different case studies. Firstly, we investigate the response of the Rosenbrock function, where the MLE algorithm shows its strengths. Secondly, we study the Griewank function, which has a challenging response surface for optimization methods. Here we see simple algorithms like the MCMC struggling to find the global optimum of the function, while algorithms like SCE-UA and DE-MCZ show their strengths. Thirdly, we apply an uncertainty analysis of a one-dimensional physically based hydrological model build with the Catchment Modelling Framework (CMF). The model is driven by meteorological and groundwater data from a Free Air Carbon Enrichment (FACE) experiment in Linden (Hesse, Germany). Simulation results are evaluated with measured soil moisture data. We search for optimal parameter sets of the van Genuchten-Mualem function and find different equally optimal solutions with some of the algorithms. The case studies reveal that the implemented SPOT methods work sufficiently well. They further show the benefit of having one tool at hand that includes a number of parameter search methods, likelihood functions and a priori parameter distributions within one platform independent package.

A Bayesian Uncertainty Framework for Conceptual Snowmelt and Hydrologic Models Applied to the Tenderfoot Creek Experimental Forest

NASA Astrophysics Data System (ADS)

Smith, T.; Marshall, L.

2007-12-01

In many mountainous regions, the single most important parameter in forecasting the controls on regional water resources is snowpack (Williams et al., 1999). In an effort to bridge the gap between theoretical understanding and functional modeling of snow-driven watersheds, a flexible hydrologic modeling framework is being developed. The aim is to create a suite of models that move from parsimonious structures, concentrated on aggregated watershed response, to those focused on representing finer scale processes and distributed response. This framework will operate as a tool to investigate the link between hydrologic model predictive performance, uncertainty, model complexity, and observable hydrologic processes. Bayesian methods, and particularly Markov chain Monte Carlo (MCMC) techniques, are extremely useful in uncertainty assessment and parameter estimation of hydrologic models. However, these methods have some difficulties in implementation. In a traditional Bayesian setting, it can be difficult to reconcile multiple data types, particularly those offering different spatial and temporal coverage, depending on the model type. These difficulties are also exacerbated by sensitivity of MCMC algorithms to model initialization and complex parameter interdependencies. As a way of circumnavigating some of the computational complications, adaptive MCMC algorithms have been developed to take advantage of the information gained from each successive iteration. Two adaptive algorithms are compared is this study, the Adaptive Metropolis (AM) algorithm, developed by Haario et al (2001), and the Delayed Rejection Adaptive Metropolis (DRAM) algorithm, developed by Haario et al (2006). While neither algorithm is truly Markovian, it has been proven that each satisfies the desired ergodicity and stationarity properties of Markov chains. Both algorithms were implemented as the uncertainty and parameter estimation framework for a conceptual rainfall-runoff model based on the Probability Distributed Model (PDM), developed by Moore (1985). We implement the modeling framework in Stringer Creek watershed in the Tenderfoot Creek Experimental Forest (TCEF), Montana. The snowmelt-driven watershed offers that additional challenge of modeling snow accumulation and melt and current efforts are aimed at developing a temperature- and radiation-index snowmelt model. Auxiliary data available from within TCEF's watersheds are used to support in the understanding of information value as it relates to predictive performance. Because the model is based on lumped parameters, auxiliary data are hard to incorporate directly. However, these additional data offer benefits through the ability to inform prior distributions of the lumped, model parameters. By incorporating data offering different information into the uncertainty assessment process, a cross-validation technique is engaged to better ensure that modeled results reflect real process complexity.

Spatiotemporal Patterns of Noise-Driven Confined Actin Waves in Living Cells.

PubMed

Bernitt, Erik; Döbereiner, Hans-Günther

2017-01-27

Cells utilize waves of polymerizing actin to reshape their morphologies, which is central to physiological and pathological processes alike. Here, we force dorsal actin waves to propagate on one-dimensional domains with periodic boundary conditions, which results in striking spatiotemporal patterns with a clear signature of noise-driven dynamics. We show that these patterns can be very closely reproduced with a noise-driven active medium at coherence resonance.

Multiscale hidden Markov models for photon-limited imaging

NASA Astrophysics Data System (ADS)

Nowak, Robert D.

1999-06-01

Photon-limited image analysis is often hindered by low signal-to-noise ratios. A novel Bayesian multiscale modeling and analysis method is developed in this paper to assist in these challenging situations. In addition to providing a very natural and useful framework for modeling an d processing images, Bayesian multiscale analysis is often much less computationally demanding compared to classical Markov random field models. This paper focuses on a probabilistic graph model called the multiscale hidden Markov model (MHMM), which captures the key inter-scale dependencies present in natural image intensities. The MHMM framework presented here is specifically designed for photon-limited imagin applications involving Poisson statistics, and applications to image intensity analysis are examined.

Linear system identification via backward-time observer models

NASA Technical Reports Server (NTRS)

Juang, Jer-Nan; Phan, Minh

1993-01-01

This paper presents an algorithm to identify a state-space model of a linear system using a backward-time approach. The procedure consists of three basic steps. First, the Markov parameters of a backward-time observer are computed from experimental input-output data. Second, the backward-time observer Markov parameters are decomposed to obtain the backward-time system Markov parameters (backward-time pulse response samples) from which a backward-time state-space model is realized using the Eigensystem Realization Algorithm. Third, the obtained backward-time state space model is converted to the usual forward-time representation. Stochastic properties of this approach will be discussed. Experimental results are given to illustrate when and to what extent this concept works.

Optimal strategies for electric energy contract decision making

NASA Astrophysics Data System (ADS)

Song, Haili

2000-10-01

The power industry restructuring in various countries in recent years has created an environment where trading of electric energy is conducted in a market environment. In such an environment, electric power companies compete for the market share through spot and bilateral markets. Being profit driven, electric power companies need to make decisions on spot market bidding, contract evaluation, and risk management. New methods and software tools are required to meet these upcoming needs. In this research, bidding strategy and contract pricing are studied from a market participant's viewpoint; new methods are developed to guide a market participant in spot and bilateral market operation. A supplier's spot market bidding decision is studied. Stochastic optimization is formulated to calculate a supplier's optimal bids in a single time period. This decision making problem is also formulated as a Markov Decision Process. All the competitors are represented by their bidding parameters with corresponding probabilities. A systematic method is developed to calculate transition probabilities and rewards. The optimal strategy is calculated to maximize the expected reward over a planning horizon. Besides the spot market, a power producer can also trade in the bilateral markets. Bidding strategies in a bilateral market are studied with game theory techniques. Necessary and sufficient conditions of Nash Equilibrium (NE) bidding strategy are derived based on the generators' cost and the loads' willingness to pay. The study shows that in any NE, market efficiency is achieved. Furthermore, all Nash equilibria are revenue equivalent for the generators. The pricing of "Flexible" contracts, which allow delivery flexibility over a period of time with a fixed total amount of electricity to be delivered, is analyzed based on the no-arbitrage pricing principle. The proposed algorithm calculates the price based on the optimality condition of the stochastic optimization formulation. Simulation examples illustrate the tradeoffs between prices and scheduling flexibility. Spot bidding and contract pricing are not independent decision processes. The interaction between spot bidding and contract evaluation is demonstrated with game theory equilibrium model and market simulation results. It leads to the conclusion that a market participant's contract decision making needs to be further investigated as an integrated optimization formulation.

Contribution of climate-driven change in continental water storage to recent sea-level rise

PubMed Central

Milly, P. C. D.; Cazenave, A.; Gennero, C.

2003-01-01

Using a global model of continental water balance, forced by interannual variations in precipitation and near-surface atmospheric temperature for the period 1981–1998, we estimate the sea-level changes associated with climate-driven changes in storage of water as snowpack, soil water, and ground water; storage in ice sheets and large lakes is not considered. The 1981–1998 trend is estimated to be 0.12 mm/yr, and substantial interannual fluctuations are inferred; for 1993–1998, the trend is 0.25 mm/yr. At the decadal time scale, the terrestrial contribution to eustatic (i.e., induced by mass exchange) sea-level rise is significantly smaller than the estimated steric (i.e., induced by density changes) trend for the same period, but is not negligibly small. In the model the sea-level rise is driven mainly by a downtrend in continental precipitation during the study period, which we believe was generated by natural variability in the climate system. PMID:14576277

Contribution of climate-driven change in continental water storage to recent sea-level rise

USGS Publications Warehouse

Milly, P.C.D.; Cazenave, A.; Gennero, M.C.

2003-01-01

Using a global model of continental water balance, forced by interannual variations in precipitation and near-surface atmospheric temperature for the period 1981-1998, we estimate the sea-level changes associated with climate-driven changes in storage of water as snowpack, soil water, and ground water; storage in ice sheets and large lakes is not considered. The 1981-1998 trend is estimated to be 0.12 mm/yr, and substantial interannual fluctuations are inferred; for 1993-1998, the trend is 0.25 mm/yr. At the decadal time scale, the terrestrial contribution to eustatic (i.e., induced by mass exchange) sea-level rise is significantly smaller than the estimated steric (i.e., induced by density changes) trend for the same period, but is not negligibly small. In the model the sea-level rise is driven mainly by a downtrend in continental precipitation during the study period, which we believe was generated by natural variability in the climate system.

Unravelling daily human mobility motifs

PubMed Central

Schneider, Christian M.; Belik, Vitaly; Couronné, Thomas; Smoreda, Zbigniew; González, Marta C.

2013-01-01

Human mobility is differentiated by time scales. While the mechanism for long time scales has been studied, the underlying mechanism on the daily scale is still unrevealed. Here, we uncover the mechanism responsible for the daily mobility patterns by analysing the temporal and spatial trajectories of thousands of persons as individual networks. Using the concept of motifs from network theory, we find only 17 unique networks are present in daily mobility and they follow simple rules. These networks, called here motifs, are sufficient to capture up to 90 per cent of the population in surveys and mobile phone datasets for different countries. Each individual exhibits a characteristic motif, which seems to be stable over several months. Consequently, daily human mobility can be reproduced by an analytically tractable framework for Markov chains by modelling periods of high-frequency trips followed by periods of lower activity as the key ingredient. PMID:23658117

Evaluation of Usability Utilizing Markov Models

ERIC Educational Resources Information Center

Penedo, Janaina Rodrigues; Diniz, Morganna; Ferreira, Simone Bacellar Leal; Silveira, Denis S.; Capra, Eliane

2012-01-01

Purpose: The purpose of this paper is to analyze the usability of a remote learning system in its initial development phase, using a quantitative usability evaluation method through Markov models. Design/methodology/approach: The paper opted for an exploratory study. The data of interest of the research correspond to the possible accesses of users…

Application of Markov Models for Analysis of Development of Psychological Characteristics

ERIC Educational Resources Information Center

Kuravsky, Lev S.; Malykh, Sergey B.

2004-01-01

A technique to study combined influence of environmental and genetic factors on the base of changes in phenotype distributions is presented. Histograms are exploited as base analyzed characteristics. A continuous time, discrete state Markov process with piece-wise constant interstate transition rates is associated with evolution of each histogram.…

Nested Fork-Join Queuing Networks and Their Application to Mobility Airfield Operations Analysis.

DTIC Science & Technology

1997-03-01

shortest queue length. Setia , Squillante, and Tripathi [109] extend Makowski and Nelson’s work by performing a quantitative assessment of a range of...Markov chains." Numerical Solution of Markov Chains, edited by W. J. Stewart, 63- 88. Basel: Marcel Dekker, 1991. [109] Setia , S. K., and others

A Test of the Need Hierarchy Concept by a Markov Model of Change in Need Strength.

ERIC Educational Resources Information Center

Rauschenberger, John; And Others

1980-01-01

In this study of 547 high school graduates, Alderfer's and Maslow's need hierarchy theories were expressed in Markov chain form and were subjected to empirical test. Both models were disconfirmed. Corroborative multiwave correlational analysis also failed to support the need hierarchy concept. (Author/IRT)

Markov Random Fields, Stochastic Quantization and Image Analysis

DTIC Science & Technology

1990-01-01

Markov random fields based on the lattice Z2 have been extensively used in image analysis in a Bayesian framework as a-priori models for the...of Image Analysis can be given some fundamental justification then there is a remarkable connection between Probabilistic Image Analysis , Statistical Mechanics and Lattice-based Euclidean Quantum Field Theory.

Teaching Markov Chain Monte Carlo: Revealing the Basic Ideas behind the Algorithm

ERIC Educational Resources Information Center

Stewart, Wayne; Stewart, Sepideh

2014-01-01

For many scientists, researchers and students Markov chain Monte Carlo (MCMC) simulation is an important and necessary tool to perform Bayesian analyses. The simulation is often presented as a mathematical algorithm and then translated into an appropriate computer program. However, this can result in overlooking the fundamental and deeper…

Modelling Faculty Replacement Strategies Using a Time-Dependent Finite Markov-Chain Process.

ERIC Educational Resources Information Center

Hackett, E. Raymond; Magg, Alexander A.; Carrigan, Sarah D.

1999-01-01

Describes the use of a time-dependent Markov-chain model to develop faculty-replacement strategies within a college at a research university. The study suggests that a stochastic modelling approach can provide valuable insight when planning for personnel needs in the immediate (five-to-ten year) future. (MSE)

Asteroid mass estimation with Markov-chain Monte Carlo

NASA Astrophysics Data System (ADS)

Siltala, L.; Granvik, M.

2017-09-01

We have developed a new Markov-chain Monte Carlo-based algorithm for asteroid mass estimation based on mutual encounters and tested it for several different asteroids. Our results are in line with previous literature values but suggest that uncertainties of prior estimates may be misleading as a consequence of using linearized methods.

Building Higher-Order Markov Chain Models with EXCEL

ERIC Educational Resources Information Center

Ching, Wai-Ki; Fung, Eric S.; Ng, Michael K.

2004-01-01

Categorical data sequences occur in many applications such as forecasting, data mining and bioinformatics. In this note, we present higher-order Markov chain models for modelling categorical data sequences with an efficient algorithm for solving the model parameters. The algorithm can be implemented easily in a Microsoft EXCEL worksheet. We give a…

Markov Processes: Exploring the Use of Dynamic Visualizations to Enhance Student Understanding

ERIC Educational Resources Information Center

Pfannkuch, Maxine; Budgett, Stephanie

2016-01-01

Finding ways to enhance introductory students' understanding of probability ideas and theory is a goal of many first-year probability courses. In this article, we explore the potential of a prototype tool for Markov processes using dynamic visualizations to develop in students a deeper understanding of the equilibrium and hitting times…

UMAP Modules-Units 105, 107-109, 111-112, 158-162.

ERIC Educational Resources Information Center

Keller, Mary K.; And Others

This collection of materials includes six units dealing with applications of matrix methods. These are: 105-Food Service Management; 107-Markov Chains; 108-Electrical Circuits; 109-Food Service and Dietary Requirements; 111-Fixed Point and Absorbing Markov Chains; and 112-Analysis of Linear Circuits. The units contain exercises and model exams,…

Development of an enhanced health-economic model and cost-effectiveness analysis of tiotropium + olodaterol Respimat® fixed-dose combination for chronic obstructive pulmonary disease patients in Italy.

PubMed

Selya-Hammer, Carl; Gonzalez-Rojas Guix, Nuria; Baldwin, Michael; Ternouth, Andrew; Miravitlles, Marc; Rutten-van Mölken, Maureen; Goosens, Lucas M A; Buyukkaramikli, Nasuh; Acciai, Valentina

2016-10-01

The objective of this study was to compare the cost-effectiveness of the fixed-dose combination (FDC) of tiotropium + olodaterol Respimat(®) FDC with tiotropium alone for patients with chronic obstructive pulmonary disease (COPD) in the Italian health care setting using a newly developed patient-level Markov model that reflects the current understanding of the disease. While previously published models have largely been based around a cohort approach using a Markov structure and GOLD stage stratification, an individual-level Markov approach was selected for the new model. Using patient-level data from the twin TOnado trials assessing Tiotropium + olodaterol Respimat(®) FDC versus tiotropium, outcomes were modelled based on the trough forced expiratory volume (tFEV1) of over 1000 patients in each treatment arm, tracked individually at trial visits through the 52-week trial period, and after the trial period it was assumed to decline at a constant rate based on disease stage. Exacerbation risk was estimated based on a random-effects logistic regression analysis of exacerbations in UPLIFT. Mortality by age and disease stage was estimated from an analysis of TIOSPIR trial data. Cost of bronchodilators and other medications, routine management, and costs of treatment for moderate and severe exacerbations for the Italian setting were included. A cost-effectiveness analysis was conducted over a 15-year time horizon from the perspective of the Italian National Health Service. Aggregating total costs and quality-adjusted life years (QALYs) for each treatment cohort over 15 years and comparing tiotropium + olodaterol Respimat(®) FDC with tiotropium alone, resulted in mean incremental costs per patient of €1167 and an incremental cost-effectiveness ratio (ICER) of €7518 per additional QALY with tiotropium + olodaterol Respimat(®) FDC. The lung function outcomes observed for tiotropium + olodaterol Respimat(®) FDC in TOnado drove the results in terms of slightly higher mean life-years (12.24 versus 12.07) exacerbation-free months (11.36 versus 11.32) per patient and slightly fewer moderate and severe exacerbations per patient-year (0.411 versus 0.415; 0.21 versus 0.24) versus tiotropium. Probabilistic sensitivity analyses showed tiotropium + olodaterol Respimat(®) FDC to be the more cost-effective treatment in 95.2% and 98.4% of 500 simulations at thresholds of €20,000 and €30,000 per QALY respectively. Tiotropium + olodaterol Respimat(®) FDC is a cost-effective bronchodilator in the maintenance treatment of COPD for the Italian health care system. © The Author(s), 2016.

The introduction of hydrogen bond and hydrophobicity effects into the rotational isomeric states model for conformational analysis of unfolded peptides.

PubMed

Engin, Ozge; Sayar, Mehmet; Erman, Burak

2009-01-13

Relative contributions of local and non-local interactions to the unfolded conformations of peptides are examined by using the rotational isomeric states model which is a Markov model based on pairwise interactions of torsion angles. The isomeric states of a residue are well described by the Ramachandran map of backbone torsion angles. The statistical weight matrices for the states are determined by molecular dynamics simulations applied to monopeptides and dipeptides. Conformational properties of tripeptides formed from combinations of alanine, valine, tyrosine and tryptophan are investigated based on the Markov model. Comparison with molecular dynamics simulation results on these tripeptides identifies the sequence-distant long-range interactions that are missing in the Markov model. These are essentially the hydrogen bond and hydrophobic interactions that are obtained between the first and the third residue of a tripeptide. A systematic correction is proposed for incorporating these long-range interactions into the rotational isomeric states model. Preliminary results suggest that the Markov assumption can be improved significantly by renormalizing the statistical weight matrices to include the effects of the long-range correlations.

Markovian Interpretations of Dual Retrieval Processes

PubMed Central

Gomes, C. F. A.; Nakamura, K.; Reyna, V. F.

2013-01-01

A half-century ago, at the dawn of the all-or-none learning era, Estes showed that finite Markov chains supply a tractable, comprehensive framework for discrete-change data of the sort that he envisioned for shifts in conditioning states in stimulus sampling theory. Shortly thereafter, such data rapidly accumulated in many spheres of human learning and animal conditioning, and Estes’ work stimulated vigorous development of Markov models to handle them. A key outcome was that the data of the workhorse paradigms of episodic memory, recognition and recall, proved to be one- and two-stage Markovian, respectively, to close approximations. Subsequently, Markov modeling of recognition and recall all but disappeared from the literature, but it is now reemerging in the wake of dual-process conceptions of episodic memory. In recall, in particular, Markov models are being used to measure two retrieval operations (direct access and reconstruction) and a slave familiarity operation. In the present paper, we develop this family of models and present the requisite machinery for fit evaluation and significance testing. Results are reviewed from selected experiments in which the recall models were used to understand dual memory processes. PMID:24948840

A comparison between MS-VECM and MS-VECMX on economic time series data

NASA Astrophysics Data System (ADS)

Phoong, Seuk-Wai; Ismail, Mohd Tahir; Sek, Siok-Kun

2014-07-01

Multivariate Markov switching models able to provide useful information on the study of structural change data since the regime switching model can analyze the time varying data and capture the mean and variance in the series of dependence structure. This paper will investigates the oil price and gold price effects on Malaysia, Singapore, Thailand and Indonesia stock market returns. Two forms of Multivariate Markov switching models are used namely the mean adjusted heteroskedasticity Markov Switching Vector Error Correction Model (MSMH-VECM) and the mean adjusted heteroskedasticity Markov Switching Vector Error Correction Model with exogenous variable (MSMH-VECMX). The reason for using these two models are to capture the transition probabilities of the data since real financial time series data always exhibit nonlinear properties such as regime switching, cointegrating relations, jumps or breaks passing the time. A comparison between these two models indicates that MSMH-VECM model able to fit the time series data better than the MSMH-VECMX model. In addition, it was found that oil price and gold price affected the stock market changes in the four selected countries.

The Markov process admits a consistent steady-state thermodynamic formalism

NASA Astrophysics Data System (ADS)

Peng, Liangrong; Zhu, Yi; Hong, Liu

2018-01-01

The search for a unified formulation for describing various non-equilibrium processes is a central task of modern non-equilibrium thermodynamics. In this paper, a novel steady-state thermodynamic formalism was established for general Markov processes described by the Chapman-Kolmogorov equation. Furthermore, corresponding formalisms of steady-state thermodynamics for the master equation and Fokker-Planck equation could be rigorously derived in mathematics. To be concrete, we proved that (1) in the limit of continuous time, the steady-state thermodynamic formalism for the Chapman-Kolmogorov equation fully agrees with that for the master equation; (2) a similar one-to-one correspondence could be established rigorously between the master equation and Fokker-Planck equation in the limit of large system size; (3) when a Markov process is restrained to one-step jump, the steady-state thermodynamic formalism for the Fokker-Planck equation with discrete state variables also goes to that for master equations, as the discretization step gets smaller and smaller. Our analysis indicated that general Markov processes admit a unified and self-consistent non-equilibrium steady-state thermodynamic formalism, regardless of underlying detailed models.

Hidden Markov models and other machine learning approaches in computational molecular biology

DOE Office of Scientific and Technical Information (OSTI.GOV)

Baldi, P.

1995-12-31

This tutorial was one of eight tutorials selected to be presented at the Third International Conference on Intelligent Systems for Molecular Biology which was held in the United Kingdom from July 16 to 19, 1995. Computational tools are increasingly needed to process the massive amounts of data, to organize and classify sequences, to detect weak similarities, to separate coding from non-coding regions, and reconstruct the underlying evolutionary history. The fundamental problem in machine learning is the same as in scientific reasoning in general, as well as statistical modeling: to come up with a good model for the data. In thismore » tutorial four classes of models are reviewed. They are: Hidden Markov models; artificial Neural Networks; Belief Networks; and Stochastic Grammars. When dealing with DNA and protein primary sequences, Hidden Markov models are one of the most flexible and powerful alignments and data base searches. In this tutorial, attention is focused on the theory of Hidden Markov Models, and how to apply them to problems in molecular biology.« less

Predicting hepatitis B monthly incidence rates using weighted Markov chains and time series methods.

PubMed

Shahdoust, Maryam; Sadeghifar, Majid; Poorolajal, Jalal; Javanrooh, Niloofar; Amini, Payam

2015-01-01

Hepatitis B (HB) is a major global mortality. Accurately predicting the trend of the disease can provide an appropriate view to make health policy disease prevention. This paper aimed to apply three different to predict monthly incidence rates of HB. This historical cohort study was conducted on the HB incidence data of Hamadan Province, the west of Iran, from 2004 to 2012. Weighted Markov Chain (WMC) method based on Markov chain theory and two time series models including Holt Exponential Smoothing (HES) and SARIMA were applied on the data. The results of different applied methods were compared to correct percentages of predicted incidence rates. The monthly incidence rates were clustered into two clusters as state of Markov chain. The correct predicted percentage of the first and second clusters for WMC, HES and SARIMA methods was (100, 0), (84, 67) and (79, 47) respectively. The overall incidence rate of HBV is estimated to decrease over time. The comparison of results of the three models indicated that in respect to existing seasonality trend and non-stationarity, the HES had the most accurate prediction of the incidence rates.

A Markov game theoretic data fusion approach for cyber situational awareness

NASA Astrophysics Data System (ADS)

Shen, Dan; Chen, Genshe; Cruz, Jose B., Jr.; Haynes, Leonard; Kruger, Martin; Blasch, Erik

2007-04-01

This paper proposes an innovative data-fusion/ data-mining game theoretic situation awareness and impact assessment approach for cyber network defense. Alerts generated by Intrusion Detection Sensors (IDSs) or Intrusion Prevention Sensors (IPSs) are fed into the data refinement (Level 0) and object assessment (L1) data fusion components. High-level situation/threat assessment (L2/L3) data fusion based on Markov game model and Hierarchical Entity Aggregation (HEA) are proposed to refine the primitive prediction generated by adaptive feature/pattern recognition and capture new unknown features. A Markov (Stochastic) game method is used to estimate the belief of each possible cyber attack pattern. Game theory captures the nature of cyber conflicts: determination of the attacking-force strategies is tightly coupled to determination of the defense-force strategies and vice versa. Also, Markov game theory deals with uncertainty and incompleteness of available information. A software tool is developed to demonstrate the performance of the high level information fusion for cyber network defense situation and a simulation example shows the enhanced understating of cyber-network defense.

The introduction of hydrogen bond and hydrophobicity effects into the rotational isomeric states model for conformational analysis of unfolded peptides

NASA Astrophysics Data System (ADS)

Engin, Ozge; Sayar, Mehmet; Erman, Burak

2009-03-01

Relative contributions of local and non-local interactions to the unfolded conformations of peptides are examined by using the rotational isomeric states model which is a Markov model based on pairwise interactions of torsion angles. The isomeric states of a residue are well described by the Ramachandran map of backbone torsion angles. The statistical weight matrices for the states are determined by molecular dynamics simulations applied to monopeptides and dipeptides. Conformational properties of tripeptides formed from combinations of alanine, valine, tyrosine and tryptophan are investigated based on the Markov model. Comparison with molecular dynamics simulation results on these tripeptides identifies the sequence-distant long-range interactions that are missing in the Markov model. These are essentially the hydrogen bond and hydrophobic interactions that are obtained between the first and the third residue of a tripeptide. A systematic correction is proposed for incorporating these long-range interactions into the rotational isomeric states model. Preliminary results suggest that the Markov assumption can be improved significantly by renormalizing the statistical weight matrices to include the effects of the long-range correlations.

Markov Task Network: A Framework for Service Composition under Uncertainty in Cyber-Physical Systems.

PubMed

Mohammed, Abdul-Wahid; Xu, Yang; Hu, Haixiao; Agyemang, Brighter

2016-09-21

In novel collaborative systems, cooperative entities collaborate services to achieve local and global objectives. With the growing pervasiveness of cyber-physical systems, however, such collaboration is hampered by differences in the operations of the cyber and physical objects, and the need for the dynamic formation of collaborative functionality given high-level system goals has become practical. In this paper, we propose a cross-layer automation and management model for cyber-physical systems. This models the dynamic formation of collaborative services pursuing laid-down system goals as an ontology-oriented hierarchical task network. Ontological intelligence provides the semantic technology of this model, and through semantic reasoning, primitive tasks can be dynamically composed from high-level system goals. In dealing with uncertainty, we further propose a novel bridge between hierarchical task networks and Markov logic networks, called the Markov task network. This leverages the efficient inference algorithms of Markov logic networks to reduce both computational and inferential loads in task decomposition. From the results of our experiments, high-precision service composition under uncertainty can be achieved using this approach.

Metadynamics Enhanced Markov Modeling of Protein Dynamics.

PubMed

Biswas, Mithun; Lickert, Benjamin; Stock, Gerhard

2018-05-31

Enhanced sampling techniques represent a versatile approach to account for rare conformational transitions in biomolecules. A particularly promising strategy is to combine massive parallel computing of short molecular dynamics (MD) trajectories (to sample the free energy landscape of the system) with Markov state modeling (to rebuild the kinetics from the sampled data). To obtain well-distributed initial structures for the short trajectories, it is proposed to employ metadynamics MD, which quickly sweeps through the entire free energy landscape of interest. Being only used to generate initial conformations, the implementation of metadynamics can be simple and fast. The conformational dynamics of helical peptide Aib 9 is adopted to discuss various technical issues of the approach, including metadynamics settings, minimal number and length of short MD trajectories, and the validation of the resulting Markov models. Using metadynamics to launch some thousands of nanosecond trajectories, several Markov state models are constructed that reveal that previous unbiased MD simulations of in total 16 μs length cannot provide correct equilibrium populations or qualitative features of the pathway distribution of the short peptide.

HIPPI: highly accurate protein family classification with ensembles of HMMs.

PubMed

Nguyen, Nam-Phuong; Nute, Michael; Mirarab, Siavash; Warnow, Tandy

2016-11-11

Given a new biological sequence, detecting membership in a known family is a basic step in many bioinformatics analyses, with applications to protein structure and function prediction and metagenomic taxon identification and abundance profiling, among others. Yet family identification of sequences that are distantly related to sequences in public databases or that are fragmentary remains one of the more difficult analytical problems in bioinformatics. We present a new technique for family identification called HIPPI (Hierarchical Profile Hidden Markov Models for Protein family Identification). HIPPI uses a novel technique to represent a multiple sequence alignment for a given protein family or superfamily by an ensemble of profile hidden Markov models computed using HMMER. An evaluation of HIPPI on the Pfam database shows that HIPPI has better overall precision and recall than blastp, HMMER, and pipelines based on HHsearch, and maintains good accuracy even for fragmentary query sequences and for protein families with low average pairwise sequence identity, both conditions where other methods degrade in accuracy. HIPPI provides accurate protein family identification and is robust to difficult model conditions. Our results, combined with observations from previous studies, show that ensembles of profile Hidden Markov models can better represent multiple sequence alignments than a single profile Hidden Markov model, and thus can improve downstream analyses for various bioinformatic tasks. Further research is needed to determine the best practices for building the ensemble of profile Hidden Markov models. HIPPI is available on GitHub at https://github.com/smirarab/sepp .

Scalable approximate policies for Markov decision process models of hospital elective admissions.

PubMed

Zhu, George; Lizotte, Dan; Hoey, Jesse

2014-05-01

To demonstrate the feasibility of using stochastic simulation methods for the solution of a large-scale Markov decision process model of on-line patient admissions scheduling. The problem of admissions scheduling is modeled as a Markov decision process in which the states represent numbers of patients using each of a number of resources. We investigate current state-of-the-art real time planning methods to compute solutions to this Markov decision process. Due to the complexity of the model, traditional model-based planners are limited in scalability since they require an explicit enumeration of the model dynamics. To overcome this challenge, we apply sample-based planners along with efficient simulation techniques that given an initial start state, generate an action on-demand while avoiding portions of the model that are irrelevant to the start state. We also propose a novel variant of a popular sample-based planner that is particularly well suited to the elective admissions problem. Results show that the stochastic simulation methods allow for the problem size to be scaled by a factor of almost 10 in the action space, and exponentially in the state space. We have demonstrated our approach on a problem with 81 actions, four specialities and four treatment patterns, and shown that we can generate solutions that are near-optimal in about 100s. Sample-based planners are a viable alternative to state-based planners for large Markov decision process models of elective admissions scheduling. Copyright © 2014 Elsevier B.V. All rights reserved.

Noise can speed convergence in Markov chains.

PubMed

Franzke, Brandon; Kosko, Bart

2011-10-01

A new theorem shows that noise can speed convergence to equilibrium in discrete finite-state Markov chains. The noise applies to the state density and helps the Markov chain explore improbable regions of the state space. The theorem ensures that a stochastic-resonance noise benefit exists for states that obey a vector-norm inequality. Such noise leads to faster convergence because the noise reduces the norm components. A corollary shows that a noise benefit still occurs if the system states obey an alternate norm inequality. This leads to a noise-benefit algorithm that requires knowledge of the steady state. An alternative blind algorithm uses only past state information to achieve a weaker noise benefit. Simulations illustrate the predicted noise benefits in three well-known Markov models. The first model is a two-parameter Ehrenfest diffusion model that shows how noise benefits can occur in the class of birth-death processes. The second model is a Wright-Fisher model of genotype drift in population genetics. The third model is a chemical reaction network of zeolite crystallization. A fourth simulation shows a convergence rate increase of 64% for states that satisfy the theorem and an increase of 53% for states that satisfy the corollary. A final simulation shows that even suboptimal noise can speed convergence if the noise applies over successive time cycles. Noise benefits tend to be sharpest in Markov models that do not converge quickly and that do not have strong absorbing states.

Operations and support cost modeling using Markov chains

NASA Technical Reports Server (NTRS)

Unal, Resit

1989-01-01

Systems for future missions will be selected with life cycle costs (LCC) as a primary evaluation criterion. This reflects the current realization that only systems which are considered affordable will be built in the future due to the national budget constaints. Such an environment calls for innovative cost modeling techniques which address all of the phases a space system goes through during its life cycle, namely: design and development, fabrication, operations and support; and retirement. A significant portion of the LCC for reusable systems are generated during the operations and support phase (OS). Typically, OS costs can account for 60 to 80 percent of the total LCC. Clearly, OS costs are wholly determined or at least strongly influenced by decisions made during the design and development phases of the project. As a result OS costs need to be considered and estimated early in the conceptual phase. To be effective, an OS cost estimating model needs to account for actual instead of ideal processes by associating cost elements with probabilities. One approach that may be suitable for OS cost modeling is the use of the Markov Chain Process. Markov chains are an important method of probabilistic analysis for operations research analysts but they are rarely used for life cycle cost analysis. This research effort evaluates the use of Markov Chains in LCC analysis by developing OS cost model for a hypothetical reusable space transportation vehicle (HSTV) and suggests further uses of the Markov Chain process as a design-aid tool.

The algebra of the general Markov model on phylogenetic trees and networks.

PubMed

Sumner, J G; Holland, B R; Jarvis, P D

2012-04-01

It is known that the Kimura 3ST model of sequence evolution on phylogenetic trees can be extended quite naturally to arbitrary split systems. However, this extension relies heavily on mathematical peculiarities of the associated Hadamard transformation, and providing an analogous augmentation of the general Markov model has thus far been elusive. In this paper, we rectify this shortcoming by showing how to extend the general Markov model on trees to include incompatible edges; and even further to more general network models. This is achieved by exploring the algebra of the generators of the continuous-time Markov chain together with the “splitting” operator that generates the branching process on phylogenetic trees. For simplicity, we proceed by discussing the two state case and then show that our results are easily extended to more states with little complication. Intriguingly, upon restriction of the two state general Markov model to the parameter space of the binary symmetric model, our extension is indistinguishable from the Hadamard approach only on trees; as soon as any incompatible splits are introduced the two approaches give rise to differing probability distributions with disparate structure. Through exploration of a simple example, we give an argument that our extension to more general networks has desirable properties that the previous approaches do not share. In particular, our construction allows for convergent evolution of previously divergent lineages; a property that is of significant interest for biological applications.

Detecting memory and structure in human navigation patterns using Markov chain models of varying order.

PubMed

Singer, Philipp; Helic, Denis; Taraghi, Behnam; Strohmaier, Markus

2014-01-01

One of the most frequently used models for understanding human navigation on the Web is the Markov chain model, where Web pages are represented as states and hyperlinks as probabilities of navigating from one page to another. Predominantly, human navigation on the Web has been thought to satisfy the memoryless Markov property stating that the next page a user visits only depends on her current page and not on previously visited ones. This idea has found its way in numerous applications such as Google's PageRank algorithm and others. Recently, new studies suggested that human navigation may better be modeled using higher order Markov chain models, i.e., the next page depends on a longer history of past clicks. Yet, this finding is preliminary and does not account for the higher complexity of higher order Markov chain models which is why the memoryless model is still widely used. In this work we thoroughly present a diverse array of advanced inference methods for determining the appropriate Markov chain order. We highlight strengths and weaknesses of each method and apply them for investigating memory and structure of human navigation on the Web. Our experiments reveal that the complexity of higher order models grows faster than their utility, and thus we confirm that the memoryless model represents a quite practical model for human navigation on a page level. However, when we expand our analysis to a topical level, where we abstract away from specific page transitions to transitions between topics, we find that the memoryless assumption is violated and specific regularities can be observed. We report results from experiments with two types of navigational datasets (goal-oriented vs. free form) and observe interesting structural differences that make a strong argument for more contextual studies of human navigation in future work.

Detecting Memory and Structure in Human Navigation Patterns Using Markov Chain Models of Varying Order

PubMed Central

Singer, Philipp; Helic, Denis; Taraghi, Behnam; Strohmaier, Markus

2014-01-01

One of the most frequently used models for understanding human navigation on the Web is the Markov chain model, where Web pages are represented as states and hyperlinks as probabilities of navigating from one page to another. Predominantly, human navigation on the Web has been thought to satisfy the memoryless Markov property stating that the next page a user visits only depends on her current page and not on previously visited ones. This idea has found its way in numerous applications such as Google's PageRank algorithm and others. Recently, new studies suggested that human navigation may better be modeled using higher order Markov chain models, i.e., the next page depends on a longer history of past clicks. Yet, this finding is preliminary and does not account for the higher complexity of higher order Markov chain models which is why the memoryless model is still widely used. In this work we thoroughly present a diverse array of advanced inference methods for determining the appropriate Markov chain order. We highlight strengths and weaknesses of each method and apply them for investigating memory and structure of human navigation on the Web. Our experiments reveal that the complexity of higher order models grows faster than their utility, and thus we confirm that the memoryless model represents a quite practical model for human navigation on a page level. However, when we expand our analysis to a topical level, where we abstract away from specific page transitions to transitions between topics, we find that the memoryless assumption is violated and specific regularities can be observed. We report results from experiments with two types of navigational datasets (goal-oriented vs. free form) and observe interesting structural differences that make a strong argument for more contextual studies of human navigation in future work. PMID:25013937

Inferring probabilistic stellar rotation periods using Gaussian processes

NASA Astrophysics Data System (ADS)

Angus, Ruth; Morton, Timothy; Aigrain, Suzanne; Foreman-Mackey, Daniel; Rajpaul, Vinesh

2018-02-01

Variability in the light curves of spotted, rotating stars is often non-sinusoidal and quasi-periodic - spots move on the stellar surface and have finite lifetimes, causing stellar flux variations to slowly shift in phase. A strictly periodic sinusoid therefore cannot accurately model a rotationally modulated stellar light curve. Physical models of stellar surfaces have many drawbacks preventing effective inference, such as highly degenerate or high-dimensional parameter spaces. In this work, we test an appropriate effective model: a Gaussian Process with a quasi-periodic covariance kernel function. This highly flexible model allows sampling of the posterior probability density function of the periodic parameter, marginalizing over the other kernel hyperparameters using a Markov Chain Monte Carlo approach. To test the effectiveness of this method, we infer rotation periods from 333 simulated stellar light curves, demonstrating that the Gaussian process method produces periods that are more accurate than both a sine-fitting periodogram and an autocorrelation function method. We also demonstrate that it works well on real data, by inferring rotation periods for 275 Kepler stars with previously measured periods. We provide a table of rotation periods for these and many more, altogether 1102 Kepler objects of interest, and their posterior probability density function samples. Because this method delivers posterior probability density functions, it will enable hierarchical studies involving stellar rotation, particularly those involving population modelling, such as inferring stellar ages, obliquities in exoplanet systems, or characterizing star-planet interactions. The code used to implement this method is available online.

Quantitative theory of driven nonlinear brain dynamics.

PubMed

Roberts, J A; Robinson, P A

2012-09-01

Strong periodic stimuli such as bright flashing lights evoke nonlinear responses in the brain and interact nonlinearly with ongoing cortical activity, but the underlying mechanisms for these phenomena are poorly understood at present. The dominant features of these experimentally observed dynamics are reproduced by the dynamics of a quantitative neural field model subject to periodic drive. Model power spectra over a range of drive frequencies show agreement with multiple features of experimental measurements, exhibiting nonlinear effects including entrainment over a range of frequencies around the natural alpha frequency f(α), subharmonic entrainment near 2f(α), and harmonic generation. Further analysis of the driven dynamics as a function of the drive parameters reveals rich nonlinear dynamics that is predicted to be observable in future experiments at high drive amplitude, including period doubling, bistable phase-locking, hysteresis, wave mixing, and chaos indicated by positive Lyapunov exponents. Moreover, photosensitive seizures are predicted for physiologically realistic model parameters yielding bistability between healthy and seizure dynamics. These results demonstrate the applicability of neural field models to the new regime of periodically driven nonlinear dynamics, enabling interpretation of experimental data in terms of specific generating mechanisms and providing new tests of the theory. Copyright © 2012 Elsevier Inc. All rights reserved.

Floquet–Magnus theory and generic transient dynamics in periodically driven many-body quantum systems

DOE Office of Scientific and Technical Information (OSTI.GOV)

Kuwahara, Tomotaka, E-mail: tomotaka.phys@gmail.com; WPI, Advanced Institute for Materials Research, Tohoku University, Sendai 980-8577; Mori, Takashi

2016-04-15

This work explores a fundamental dynamical structure for a wide range of many-body quantum systems under periodic driving. Generically, in the thermodynamic limit, such systems are known to heat up to infinite temperature states in the long-time limit irrespective of dynamical details, which kills all the specific properties of the system. In the present study, instead of considering infinitely long-time scale, we aim to provide a general framework to understand the long but finite time behavior, namely the transient dynamics. In our analysis, we focus on the Floquet–Magnus (FM) expansion that gives a formal expression of the effective Hamiltonian onmore » the system. Although in general the full series expansion is not convergent in the thermodynamics limit, we give a clear relationship between the FM expansion and the transient dynamics. More precisely, we rigorously show that a truncated version of the FM expansion accurately describes the exact dynamics for a certain time-scale. Our theory reveals an experimental time-scale for which non-trivial dynamical phenomena can be reliably observed. We discuss several dynamical phenomena, such as the effect of small integrability breaking, efficient numerical simulation of periodically driven systems, dynamical localization and thermalization. Especially on thermalization, we discuss a generic scenario on the prethermalization phenomenon in periodically driven systems. -- Highlights: •A general framework to describe transient dynamics for periodically driven systems. •The theory is applicable to generic quantum many-body systems including long-range interacting systems. •Physical meaning of the truncation of the Floquet–Magnus expansion is rigorously established. •New mechanism of the prethermalization is proposed. •Revealing an experimental time-scale for which non-trivial dynamical phenomena can be reliably observed.« less

Flow-Driven Waves and Phase-Locked Self-Organization in Quasi-One-Dimensional Colonies of Dictyostelium discoideum

NASA Astrophysics Data System (ADS)

Gholami, A.; Steinbock, O.; Zykov, V.; Bodenschatz, E.

2015-01-01

We report experiments on flow-driven waves in a microfluidic channel containing the signaling slime mold Dictyostelium discoideum. The observed cyclic adenosine monophosphate (cAMP) wave trains developed spontaneously in the presence of flow and propagated with the velocity proportional to the imposed flow velocity. The period of the wave trains was independent of the flow velocity. Perturbations of flow-driven waves via external periodic pulses of the signaling agent cAMP induced 1 ∶1 , 2 ∶1 , 3 ∶1 , and 1 ∶2 frequency responses, reminiscent of Arnold tongues in forced oscillatory systems. We expect our observations to be generic to active media governed by reaction-diffusion-advection dynamics, where spatially bound autocatalytic processes occur under flow conditions.

Experiences with Markov Chain Monte Carlo Convergence Assessment in Two Psychometric Examples

ERIC Educational Resources Information Center

Sinharay, Sandip

2004-01-01

There is an increasing use of Markov chain Monte Carlo (MCMC) algorithms for fitting statistical models in psychometrics, especially in situations where the traditional estimation techniques are very difficult to apply. One of the disadvantages of using an MCMC algorithm is that it is not straightforward to determine the convergence of the…

Tracking Problem Solving by Multivariate Pattern Analysis and Hidden Markov Model Algorithms

ERIC Educational Resources Information Center

Anderson, John R.

2012-01-01

Multivariate pattern analysis can be combined with Hidden Markov Model algorithms to track the second-by-second thinking as people solve complex problems. Two applications of this methodology are illustrated with a data set taken from children as they interacted with an intelligent tutoring system for algebra. The first "mind reading" application…

A Markov Chain Monte Carlo Approach to Confirmatory Item Factor Analysis

ERIC Educational Resources Information Center

Edwards, Michael C.

2010-01-01

Item factor analysis has a rich tradition in both the structural equation modeling and item response theory frameworks. The goal of this paper is to demonstrate a novel combination of various Markov chain Monte Carlo (MCMC) estimation routines to estimate parameters of a wide variety of confirmatory item factor analysis models. Further, I show…

Markov Chain Monte Carlo Estimation of Item Parameters for the Generalized Graded Unfolding Model

ERIC Educational Resources Information Center

de la Torre, Jimmy; Stark, Stephen; Chernyshenko, Oleksandr S.

2006-01-01

The authors present a Markov Chain Monte Carlo (MCMC) parameter estimation procedure for the generalized graded unfolding model (GGUM) and compare it to the marginal maximum likelihood (MML) approach implemented in the GGUM2000 computer program, using simulated and real personality data. In the simulation study, test length, number of response…

Recovery of Graded Response Model Parameters: A Comparison of Marginal Maximum Likelihood and Markov Chain Monte Carlo Estimation

ERIC Educational Resources Information Center

Kieftenbeld, Vincent; Natesan, Prathiba

2012-01-01

Markov chain Monte Carlo (MCMC) methods enable a fully Bayesian approach to parameter estimation of item response models. In this simulation study, the authors compared the recovery of graded response model parameters using marginal maximum likelihood (MML) and Gibbs sampling (MCMC) under various latent trait distributions, test lengths, and…

Chutes and Ladders for the Impatient

ERIC Educational Resources Information Center

Cheteyan, Leslie A.; Hengeveld, Stewart; Jones, Michael A.

2011-01-01

In this paper, we review the rules and game board for "Chutes and Ladders", define a Markov chain to model the game regardless of the spinner range, and describe how properties of Markov chains are used to determine that an optimal spinner range of 15 minimizes the expected number of turns for a player to complete the game. Because the Markov…

Stability of the Markov operator and synchronization of Markovian random products

NASA Astrophysics Data System (ADS)

Díaz, Lorenzo J.; Matias, Edgar

2018-05-01

We study Markovian random products on a large class of ‘m-dimensional’ connected compact metric spaces (including products of closed intervals and trees). We introduce a splitting condition, generalizing the classical one by Dubins and Freedman, and prove that this condition implies the asymptotic stability of the corresponding Markov operator and (exponentially fast) synchronization.

Generalization of Faustmann's Formula for Stochastic Forest Growth and Prices with Markov Decision Process Models

Treesearch

Joseph Buongiorno

2001-01-01

Faustmann's formula gives the land value, or the forest value of land with trees, under deterministic assumptions regarding future stand growth and prices, over an infinite horizon. Markov decision process (MDP) models generalize Faustmann's approach by recognizing that future stand states and prices are known only as probabilistic distributions. The...

Students' Progress throughout Examination Process as a Markov Chain

ERIC Educational Resources Information Center

Hlavatý, Robert; Dömeová, Ludmila

2014-01-01

The paper is focused on students of Mathematical methods in economics at the Czech university of life sciences (CULS) in Prague. The idea is to create a model of students' progress throughout the whole course using the Markov chain approach. Each student has to go through various stages of the course requirements where his success depends on the…

Tracking Skill Acquisition with Cognitive Diagnosis Models: A Higher-Order, Hidden Markov Model with Covariates

ERIC Educational Resources Information Center

Wang, Shiyu; Yang, Yan; Culpepper, Steven Andrew; Douglas, Jeffrey A.

2018-01-01

A family of learning models that integrates a cognitive diagnostic model and a higher-order, hidden Markov model in one framework is proposed. This new framework includes covariates to model skill transition in the learning environment. A Bayesian formulation is adopted to estimate parameters from a learning model. The developed methods are…

The distribution of genome shared identical by descent for a pair of full sibs by means of the continuous time Markov chain

NASA Astrophysics Data System (ADS)

Julie, Hongki; Pasaribu, Udjianna S.; Pancoro, Adi

2015-12-01

This paper will allow Markov Chain's application in genome shared identical by descent by two individual at full sibs model. The full sibs model was a continuous time Markov Chain with three state. In the full sibs model, we look for the cumulative distribution function of the number of sub segment which have 2 IBD haplotypes from a segment of the chromosome which the length is t Morgan and the cumulative distribution function of the number of sub segment which have at least 1 IBD haplotypes from a segment of the chromosome which the length is t Morgan. This cumulative distribution function will be developed by the moment generating function.

MARKOV: A methodology for the solution of infinite time horizon MARKOV decision processes

USGS Publications Warehouse

Williams, B.K.

1988-01-01

Algorithms are described for determining optimal policies for finite state, finite action, infinite discrete time horizon Markov decision processes. Both value-improvement and policy-improvement techniques are used in the algorithms. Computing procedures are also described. The algorithms are appropriate for processes that are either finite or infinite, deterministic or stochastic, discounted or undiscounted, in any meaningful combination of these features. Computing procedures are described in terms of initial data processing, bound improvements, process reduction, and testing and solution. Application of the methodology is illustrated with an example involving natural resource management. Management implications of certain hypothesized relationships between mallard survival and harvest rates are addressed by applying the optimality procedures to mallard population models.

Techniques for modeling the reliability of fault-tolerant systems with the Markov state-space approach

NASA Technical Reports Server (NTRS)

Butler, Ricky W.; Johnson, Sally C.

1995-01-01

This paper presents a step-by-step tutorial of the methods and the tools that were used for the reliability analysis of fault-tolerant systems. The approach used in this paper is the Markov (or semi-Markov) state-space method. The paper is intended for design engineers with a basic understanding of computer architecture and fault tolerance, but little knowledge of reliability modeling. The representation of architectural features in mathematical models is emphasized. This paper does not present details of the mathematical solution of complex reliability models. Instead, it describes the use of several recently developed computer programs SURE, ASSIST, STEM, and PAWS that automate the generation and the solution of these models.

Lindeberg theorem for Gibbs-Markov dynamics

NASA Astrophysics Data System (ADS)

Denker, Manfred; Senti, Samuel; Zhang, Xuan

2017-12-01

A dynamical array consists of a family of functions \\{ fn, i: 1≤slant i≤slant k_n, n≥slant 1\\} and a family of initial times \\{τn, i: 1≤slant i≤slant k_n, n≥slant 1\\} . For a dynamical system (X, T) we identify distributional limits for sums of the form for suitable (non-random) constants s_n>0 and an, i\\in { R} . We derive a Lindeberg-type central limit theorem for dynamical arrays. Applications include new central limit theorems for functions which are not locally Lipschitz continuous and central limit theorems for statistical functions of time series obtained from Gibbs-Markov systems. Our results, which hold for more general dynamics, are stated in the context of Gibbs-Markov dynamical systems for convenience.

Discrete time Markov chains (DTMC) susceptible infected susceptible (SIS) epidemic model with two pathogens in two patches

NASA Astrophysics Data System (ADS)

Lismawati, Eka; Respatiwulan; Widyaningsih, Purnami

2017-06-01

The SIS epidemic model describes the pattern of disease spread with characteristics that recovered individuals can be infected more than once. The number of susceptible and infected individuals every time follows the discrete time Markov process. It can be represented by the discrete time Markov chains (DTMC) SIS. The DTMC SIS epidemic model can be developed for two pathogens in two patches. The aims of this paper are to reconstruct and to apply the DTMC SIS epidemic model with two pathogens in two patches. The model was presented as transition probabilities. The application of the model obtain that the number of susceptible individuals decreases while the number of infected individuals increases for each pathogen in each patch.

Simplification of irreversible Markov chains by removal of states with fast leaving rates.

PubMed

Jia, Chen

2016-07-07

In the recent work of Ullah et al. (2012a), the authors developed an effective method to simplify reversible Markov chains by removal of states with low equilibrium occupancies. In this paper, we extend this result to irreversible Markov chains. We show that an irreversible chain can be simplified by removal of states with fast leaving rates. Moreover, we reveal that the irreversibility of the chain will always decrease after model simplification. This suggests that although model simplification can retain almost all the dynamic information of the chain, it will lose some thermodynamic information as a trade-off. Examples from biology are also given to illustrate the main results of this paper. Copyright © 2016 Elsevier Ltd. All rights reserved.

Network Security Risk Assessment System Based on Attack Graph and Markov Chain

NASA Astrophysics Data System (ADS)

Sun, Fuxiong; Pi, Juntao; Lv, Jin; Cao, Tian

2017-10-01

Network security risk assessment technology can be found in advance of the network problems and related vulnerabilities, it has become an important means to solve the problem of network security. Based on attack graph and Markov chain, this paper provides a Network Security Risk Assessment Model (NSRAM). Based on the network infiltration tests, NSRAM generates the attack graph by the breadth traversal algorithm. Combines with the international standard CVSS, the attack probability of atomic nodes are counted, and then the attack transition probabilities of ones are calculated by Markov chain. NSRAM selects the optimal attack path after comprehensive measurement to assessment network security risk. The simulation results show that NSRAM can reflect the actual situation of network security objectively.

An 'adding' algorithm for the Markov chain formalism for radiation transfer

NASA Technical Reports Server (NTRS)

Esposito, L. W.

1979-01-01

An adding algorithm is presented, that extends the Markov chain method and considers a preceding calculation as a single state of a new Markov chain. This method takes advantage of the description of the radiation transport as a stochastic process. Successive application of this procedure makes calculation possible for any optical depth without increasing the size of the linear system used. It is determined that the time required for the algorithm is comparable to that for a doubling calculation for homogeneous atmospheres. For an inhomogeneous atmosphere the new method is considerably faster than the standard adding routine. It is concluded that the algorithm is efficient, accurate, and suitable for smaller computers in calculating the diffuse intensity scattered by an inhomogeneous planetary atmosphere.

Metastable Distributions of Markov Chains with Rare Transitions

NASA Astrophysics Data System (ADS)

Freidlin, M.; Koralov, L.

2017-06-01

In this paper we consider Markov chains X^\\varepsilon _t with transition rates that depend on a small parameter \\varepsilon . We are interested in the long time behavior of X^\\varepsilon _t at various \\varepsilon -dependent time scales t = t(\\varepsilon ). The asymptotic behavior depends on how the point (1/\\varepsilon , t(\\varepsilon )) approaches infinity. We introduce a general notion of complete asymptotic regularity (a certain asymptotic relation between the ratios of transition rates), which ensures the existence of the metastable distribution for each initial point and a given time scale t(\\varepsilon ). The technique of i-graphs allows one to describe the metastable distribution explicitly. The result may be viewed as a generalization of the ergodic theorem to the case of parameter-dependent Markov chains.

A big-data model for multi-modal public transportation with application to macroscopic control and optimisation

NASA Astrophysics Data System (ADS)

Faizrahnemoon, Mahsa; Schlote, Arieh; Maggi, Lorenzo; Crisostomi, Emanuele; Shorten, Robert

2015-11-01

This paper describes a Markov-chain-based approach to modelling multi-modal transportation networks. An advantage of the model is the ability to accommodate complex dynamics and handle huge amounts of data. The transition matrix of the Markov chain is built and the model is validated using the data extracted from a traffic simulator. A realistic test-case using multi-modal data from the city of London is given to further support the ability of the proposed methodology to handle big quantities of data. Then, we use the Markov chain as a control tool to improve the overall efficiency of a transportation network, and some practical examples are described to illustrate the potentials of the approach.

GENERAL: A Possible Population-Driven Phase Transition in Cicada Chorus

NASA Astrophysics Data System (ADS)

Gu, Si-Yuan; Jin, Yu-Liang; Zhao, Xiao-Xue; Huang, Ji-Ping

2009-06-01

We investigate the collective synchronization of cicada chirping. Using both experimental and phenomenological numerical techniques, here we show that the onset of a periodic two-state acoustic synchronous behavior in cicada chorus depends on a critical size of population Nc = 21, above which a typical chorus state appears periodically with a 30 second-silence state in between, and further clarify its possibility concerning a new class of phase transition, which is unusually driven by population. This work has relevance to acoustic synchronization and to general physics of phase transition.

Reconstructing Tropical Pacific Sea Level Variability for the Period 1961-2002 Using a Linear Multimode Model

NASA Astrophysics Data System (ADS)

Greatbatch, Richard J.; Zhu, Xiaoting; Claus, Martin

2018-03-01

Monthly mean sea level anomalies in the tropical Pacific for the period 1961-2002 are reconstructed using a linear, multimode model driven by monthly mean wind stress anomalies from the NCEP/NCAR and ERA-40 reanalysis products. Overall, the sea level anomalies reconstructed by both wind stress products agree well with the available tide gauge data, although with poor performance at Kanton Island in the western-central equatorial Pacific and reduced amplitude at Christmas Island. The reduced performance is related to model error in locating the pivot point in sea level variability associated with the so-called "tilt" mode. We present evidence that the pivot point was further west during the period 1993-2014 than during the period 1961-2002 and attribute this to a persistent upward trend in the zonal wind stress variance along the equator west of 160° W throughout the period 1961-2014. Experiments driven by the zonal component of the wind stress alone reproduce much of the trend in sea level found in the experiments driven by both components of the wind stress. The experiments show an upward trend in sea level in the eastern tropical Pacific over the period 1961-2002, but with a much stronger upward trend when using the NCEP/NCAR product. We argue that the latter is related to an overly strong eastward trend in zonal wind stress in the eastern-central Pacific that is believed to be a spurious feature of the NCEP/NCAR product.

Detecting seismic waves using a binary hidden Markov model classifier

NASA Astrophysics Data System (ADS)

Ray, J.; Lefantzi, S.; Brogan, R. A.; Forrest, R.; Hansen, C. W.; Young, C. J.

2016-12-01

We explore the use of Hidden Markov Models (HMM) to detect the arrival of seismic waves using data captured by a seismogram. HMMs define the state of a station as a binary variable based on whether the station is receiving a signal or not. HMMs are simple and fast, allowing them to monitor multiple datastreams arising from a large distributed network of seismographs. In this study we examine the efficacy of HMM-based detectors with respect to their false positive and negative rates as well as the accuracy of the signal onset time as compared to the value determined by an expert analyst. The study uses 3 component International Monitoring System (IMS) data from a carefully analyzed 2 week period from May, 2010, for which our analyst tried to identify every signal. Part of this interval is used for training the HMM to recognize the transition between state from noise to signal, while the other is used for evaluating the effectiveness of our new detection algorithm. We compare our results with the STA/LTA detection processing applied by the IDC to assess potential for operational use. Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the U.S. Department of Energy's National Nuclear Security Administration under contract DE-AC04-94AL85000.

Nonstationary Extreme Value Analysis in a Changing Climate: A Software Package

NASA Astrophysics Data System (ADS)

Cheng, L.; AghaKouchak, A.; Gilleland, E.

2013-12-01

Numerous studies show that climatic extremes have increased substantially in the second half of the 20th century. For this reason, analysis of extremes under a nonstationary assumption has received a great deal of attention. This paper presents a software package developed for estimation of return levels, return periods, and risks of climatic extremes in a changing climate. This MATLAB software package offers tools for analysis of climate extremes under both stationary and non-stationary assumptions. The Nonstationary Extreme Value Analysis (hereafter, NEVA) provides an efficient and generalized framework for analyzing extremes using Bayesian inference. NEVA estimates the extreme value parameters using a Differential Evolution Markov Chain (DE-MC) which utilizes the genetic algorithm Differential Evolution (DE) for global optimization over the real parameter space with the Markov Chain Monte Carlo (MCMC) approach and has the advantage of simplicity, speed of calculation and convergence over conventional MCMC. NEVA also offers the confidence interval and uncertainty bounds of estimated return levels based on the sampled parameters. NEVA integrates extreme value design concepts, data analysis tools, optimization and visualization, explicitly designed to facilitate analysis extremes in geosciences. The generalized input and output files of this software package make it attractive for users from across different fields. Both stationary and nonstationary components of the package are validated for a number of case studies using empirical return levels. The results show that NEVA reliably describes extremes and their return levels.

Pharmacist-driven antimicrobial optimization in the emergency department.

PubMed

Davis, Lucretia C; Covey, Robin B; Weston, Jaye S; Hu, Bee Bee Y; Laine, Gregory A

2016-03-01

A pharmacist-driven antimicrobial optimization service in the non-trauma emergency department (ED) of an 864-bed non-profit tertiary care teaching hospital was reviewed to assess its value. Local antimicrobial resistance patterns of urine, wound, stool, and blood cultures were also studied to determine whether or not empiric prescribing practices should be modified. A retrospective electronic chart review was performed for ED patients with positive cultures during two different three-month periods. During Period 1, ED nursing management performed positive culture follow-up. During Period 2, ED clinical pharmacists performed this role. The primary objective was to determine the value of the pharmacist-driven antimicrobial optimization service as measured by the number of clinical interventions made when indicated. The secondary objective was to examine resistance patterns of urine and wound isolates in order to determine if empiric prescribing patterns in the ED should be modified. During Period 1, there were 499 patient visits with subsequent positive cultures. Of those, 76 patients (15%) were discharged home. Nursing management intervened on 21 of 42 (50%) positive cultures that required an intervention; in Period 2, there were 473 patient visits with subsequent positive cultures, and 64 (14%) were discharged home. Pharmacists intervened on 24 of 30 (80%) cultures where an intervention was indicated resulting in a 30% increase in interventions for inappropriate therapy (p = 0.01). A review of the secondary objective revealed a 38% fluoroquinolone resistance rate of E. coli, the most frequently isolated urinary organism. Pharmacist-driven antimicrobial stewardship program resulted in a 30% absolute increase in interventions for inappropriate therapy as compared to the nursing-driven model. This stewardship program has further demonstrated the value of ED pharmacists. Pharmacist interventions should help to ensure that infections are resolved through modification of antimicrobial therapies for patients with bug-drug mismatches. The fluoroquinolone resistance rate indicates a need to consider alternative therapies for uncomplicated urinary tract infections. Nitrofurantoin remains with good coverage against E. coli and Enterococcus species but should be used in uncomplicated patients with normal renal function. Copyright © 2016 by the American Society of Health-System Pharmacists, Inc. All rights reserved.

Markov chain decision model for urinary incontinence procedures.

PubMed

Kumar, Sameer; Ghildayal, Nidhi; Ghildayal, Neha

2017-03-13

Purpose Urinary incontinence (UI) is a common chronic health condition, a problem specifically among elderly women that impacts quality of life negatively. However, UI is usually viewed as likely result of old age, and as such is generally not evaluated or even managed appropriately. Many treatments are available to manage incontinence, such as bladder training and numerous surgical procedures such as Burch colposuspension and Sling for UI which have high success rates. The purpose of this paper is to analyze which of these popular surgical procedures for UI is effective. Design/methodology/approach This research employs randomized, prospective studies to obtain robust cost and utility data used in the Markov chain decision model for examining which of these surgical interventions is more effective in treating women with stress UI based on two measures: number of quality adjusted life years (QALY) and cost per QALY. Treeage Pro Healthcare software was employed in Markov decision analysis. Findings Results showed the Sling procedure is a more effective surgical intervention than the Burch. However, if a utility greater than certain utility value, for which both procedures are equally effective, is assigned to persistent incontinence, the Burch procedure is more effective than the Sling procedure. Originality/value This paper demonstrates the efficacy of a Markov chain decision modeling approach to study the comparative effectiveness analysis of available treatments for patients with UI, an important public health issue, widely prevalent among elderly women in developed and developing countries. This research also improves upon other analyses using a Markov chain decision modeling process to analyze various strategies for treating UI.

Development and validation of a Markov microsimulation model for the economic evaluation of treatments in osteoporosis.

PubMed

Hiligsmann, Mickaël; Ethgen, Olivier; Bruyère, Olivier; Richy, Florent; Gathon, Henry-Jean; Reginster, Jean-Yves

2009-01-01

Markov models are increasingly used in economic evaluations of treatments for osteoporosis. Most of the existing evaluations are cohort-based Markov models missing comprehensive memory management and versatility. In this article, we describe and validate an original Markov microsimulation model to accurately assess the cost-effectiveness of prevention and treatment of osteoporosis. We developed a Markov microsimulation model with a lifetime horizon and a direct health-care cost perspective. The patient history was recorded and was used in calculations of transition probabilities, utilities, and costs. To test the internal consistency of the model, we carried out an example calculation for alendronate therapy. Then, external consistency was investigated by comparing absolute lifetime risk of fracture estimates with epidemiologic data. For women at age 70 years, with a twofold increase in the fracture risk of the average population, the costs per quality-adjusted life-year gained for alendronate therapy versus no treatment were estimated at €9105 and €15,325, respectively, under full and realistic adherence assumptions. All the sensitivity analyses in terms of model parameters and modeling assumptions were coherent with expected conclusions and absolute lifetime risk of fracture estimates were within the range of previous estimates, which confirmed both internal and external consistency of the model. Microsimulation models present some major advantages over cohort-based models, increasing the reliability of the results and being largely compatible with the existing state of the art, evidence-based literature. The developed model appears to be a valid model for use in economic evaluations in osteoporosis.

A response to Yu et al. "A forward-backward fragment assembling algorithm for the identification of genomic amplification and deletion breakpoints using high-density single nucleotide polymorphism (SNP) array", BMC Bioinformatics 2007, 8: 145.

PubMed

Rueda, Oscar M; Diaz-Uriarte, Ramon

2007-10-16

Yu et al. (BMC Bioinformatics 2007,8: 145+) have recently compared the performance of several methods for the detection of genomic amplification and deletion breakpoints using data from high-density single nucleotide polymorphism arrays. One of the methods compared is our non-homogenous Hidden Markov Model approach. Our approach uses Markov Chain Monte Carlo for inference, but Yu et al. ran the sampler for a severely insufficient number of iterations for a Markov Chain Monte Carlo-based method. Moreover, they did not use the appropriate reference level for the non-altered state. We rerun the analysis in Yu et al. using appropriate settings for both the Markov Chain Monte Carlo iterations and the reference level. Additionally, to show how easy it is to obtain answers to additional specific questions, we have added a new analysis targeted specifically to the detection of breakpoints. The reanalysis shows that the performance of our method is comparable to that of the other methods analyzed. In addition, we can provide probabilities of a given spot being a breakpoint, something unique among the methods examined. Markov Chain Monte Carlo methods require using a sufficient number of iterations before they can be assumed to yield samples from the distribution of interest. Running our method with too small a number of iterations cannot be representative of its performance. Moreover, our analysis shows how our original approach can be easily adapted to answer specific additional questions (e.g., identify edges).

The application of a Grey Markov Model to forecasting annual maximum water levels at hydrological stations

NASA Astrophysics Data System (ADS)

Dong, Sheng; Chi, Kun; Zhang, Qiyi; Zhang, Xiangdong

2012-03-01

Compared with traditional real-time forecasting, this paper proposes a Grey Markov Model (GMM) to forecast the maximum water levels at hydrological stations in the estuary area. The GMM combines the Grey System and Markov theory into a higher precision model. The GMM takes advantage of the Grey System to predict the trend values and uses the Markov theory to forecast fluctuation values, and thus gives forecast results involving two aspects of information. The procedure for forecasting annul maximum water levels with the GMM contains five main steps: 1) establish the GM (1, 1) model based on the data series; 2) estimate the trend values; 3) establish a Markov Model based on relative error series; 4) modify the relative errors caused in step 2, and then obtain the relative errors of the second order estimation; 5) compare the results with measured data and estimate the accuracy. The historical water level records (from 1960 to 1992) at Yuqiao Hydrological Station in the estuary area of the Haihe River near Tianjin, China are utilized to calibrate and verify the proposed model according to the above steps. Every 25 years' data are regarded as a hydro-sequence. Eight groups of simulated results show reasonable agreement between the predicted values and the measured data. The GMM is also applied to the 10 other hydrological stations in the same estuary. The forecast results for all of the hydrological stations are good or acceptable. The feasibility and effectiveness of this new forecasting model have been proved in this paper.

An Evaluation of a Markov Chain Monte Carlo Method for the Two-Parameter Logistic Model.

ERIC Educational Resources Information Center

Kim, Seock-Ho; Cohen, Allan S.

The accuracy of the Markov Chain Monte Carlo (MCMC) procedure Gibbs sampling was considered for estimation of item parameters of the two-parameter logistic model. Data for the Law School Admission Test (LSAT) Section 6 were analyzed to illustrate the MCMC procedure. In addition, simulated data sets were analyzed using the MCMC, marginal Bayesian…

Housing Value Projection Model Related to Educational Planning: The Feasibility of a New Methodology. Final Report.

ERIC Educational Resources Information Center

Helbock, Richard W.; Marker, Gordon

This study concerns the feasibility of a Markov chain model for projecting housing values and racial mixes. Such projections could be used in planning the layout of school districts to achieve desired levels of socioeconomic heterogeneity. Based upon the concepts and assumptions underlying a Markov chain model, it is concluded that such a model is…

Causal Latent Markov Model for the Comparison of Multiple Treatments in Observational Longitudinal Studies

ERIC Educational Resources Information Center

Bartolucci, Francesco; Pennoni, Fulvia; Vittadini, Giorgio

2016-01-01

We extend to the longitudinal setting a latent class approach that was recently introduced by Lanza, Coffman, and Xu to estimate the causal effect of a treatment. The proposed approach enables an evaluation of multiple treatment effects on subpopulations of individuals from a dynamic perspective, as it relies on a latent Markov (LM) model that is…

Recovery of Item Parameters in the Nominal Response Model: A Comparison of Marginal Maximum Likelihood Estimation and Markov Chain Monte Carlo Estimation.

ERIC Educational Resources Information Center

Wollack, James A.; Bolt, Daniel M.; Cohen, Allan S.; Lee, Young-Sun

2002-01-01

Compared the quality of item parameter estimates for marginal maximum likelihood (MML) and Markov Chain Monte Carlo (MCMC) with the nominal response model using simulation. The quality of item parameter recovery was nearly identical for MML and MCMC, and both methods tended to produce good estimates. (SLD)

Markov Chains For Testing Redundant Software

NASA Technical Reports Server (NTRS)

White, Allan L.; Sjogren, Jon A.

1990-01-01

Preliminary design developed for validation experiment that addresses problems unique to assuring extremely high quality of multiple-version programs in process-control software. Approach takes into account inertia of controlled system in sense it takes more than one failure of control program to cause controlled system to fail. Verification procedure consists of two steps: experimentation (numerical simulation) and computation, with Markov model for each step.

An NCME Instructional Module on Estimating Item Response Theory Models Using Markov Chain Monte Carlo Methods

ERIC Educational Resources Information Center

Kim, Jee-Seon; Bolt, Daniel M.

2007-01-01

The purpose of this ITEMS module is to provide an introduction to Markov chain Monte Carlo (MCMC) estimation for item response models. A brief description of Bayesian inference is followed by an overview of the various facets of MCMC algorithms, including discussion of prior specification, sampling procedures, and methods for evaluating chain…

Hidden Markov models for character recognition.

PubMed

Vlontzos, J A; Kung, S Y

1992-01-01

A hierarchical system for character recognition with hidden Markov model knowledge sources which solve both the context sensitivity problem and the character instantiation problem is presented. The system achieves 97-99% accuracy using a two-level architecture and has been implemented using a systolic array, thus permitting real-time (1 ms per character) multifont and multisize printed character recognition as well as handwriting recognition.