Recurrent Network models of sequence generation and memory

PubMed Central

Rajan, Kanaka; Harvey, Christopher D; Tank, David W

2016-01-01

SUMMARY Sequential activation of neurons is a common feature of network activity during a variety of behaviors, including working memory and decision making. Previous network models for sequences and memory emphasized specialized architectures in which a principled mechanism is pre-wired into their connectivity. Here, we demonstrate that starting from random connectivity and modifying a small fraction of connections, a largely disordered recurrent network can produce sequences and implement working memory efficiently. We use this process, called Partial In-Network training (PINning), to model and match cellular-resolution imaging data from the posterior parietal cortex during a virtual memory-guided two-alternative forced choice task [Harvey, Coen and Tank, 2012]. Analysis of the connectivity reveals that sequences propagate by the cooperation between recurrent synaptic interactions and external inputs, rather than through feedforward or asymmetric connections. Together our results suggest that neural sequences may emerge through learning from largely unstructured network architectures. PMID:26971945

Reduced-Order Modeling for Flutter/LCO Using Recurrent Artificial Neural Network

NASA Technical Reports Server (NTRS)

Yao, Weigang; Liou, Meng-Sing

2012-01-01

The present study demonstrates the efficacy of a recurrent artificial neural network to provide a high fidelity time-dependent nonlinear reduced-order model (ROM) for flutter/limit-cycle oscillation (LCO) modeling. An artificial neural network is a relatively straightforward nonlinear method for modeling an input-output relationship from a set of known data, for which we use the radial basis function (RBF) with its parameters determined through a training process. The resulting RBF neural network, however, is only static and is not yet adequate for an application to problems of dynamic nature. The recurrent neural network method [1] is applied to construct a reduced order model resulting from a series of high-fidelity time-dependent data of aero-elastic simulations. Once the RBF neural network ROM is constructed properly, an accurate approximate solution can be obtained at a fraction of the cost of a full-order computation. The method derived during the study has been validated for predicting nonlinear aerodynamic forces in transonic flow and is capable of accurate flutter/LCO simulations. The obtained results indicate that the present recurrent RBF neural network is accurate and efficient for nonlinear aero-elastic system analysis

Recurrence network measures for hypothesis testing using surrogate data: Application to black hole light curves

NASA Astrophysics Data System (ADS)

Jacob, Rinku; Harikrishnan, K. P.; Misra, R.; Ambika, G.

2018-01-01

Recurrence networks and the associated statistical measures have become important tools in the analysis of time series data. In this work, we test how effective the recurrence network measures are in analyzing real world data involving two main types of noise, white noise and colored noise. We use two prominent network measures as discriminating statistic for hypothesis testing using surrogate data for a specific null hypothesis that the data is derived from a linear stochastic process. We show that the characteristic path length is especially efficient as a discriminating measure with the conclusions reasonably accurate even with limited number of data points in the time series. We also highlight an additional advantage of the network approach in identifying the dimensionality of the system underlying the time series through a convergence measure derived from the probability distribution of the local clustering coefficients. As examples of real world data, we use the light curves from a prominent black hole system and show that a combined analysis using three primary network measures can provide vital information regarding the nature of temporal variability of light curves from different spectroscopic classes.

Financial Time Series Prediction Using Elman Recurrent Random Neural Networks

PubMed Central

Wang, Jie; Wang, Jun; Fang, Wen; Niu, Hongli

2016-01-01

In recent years, financial market dynamics forecasting has been a focus of economic research. To predict the price indices of stock markets, we developed an architecture which combined Elman recurrent neural networks with stochastic time effective function. By analyzing the proposed model with the linear regression, complexity invariant distance (CID), and multiscale CID (MCID) analysis methods and taking the model compared with different models such as the backpropagation neural network (BPNN), the stochastic time effective neural network (STNN), and the Elman recurrent neural network (ERNN), the empirical results show that the proposed neural network displays the best performance among these neural networks in financial time series forecasting. Further, the empirical research is performed in testing the predictive effects of SSE, TWSE, KOSPI, and Nikkei225 with the established model, and the corresponding statistical comparisons of the above market indices are also exhibited. The experimental results show that this approach gives good performance in predicting the values from the stock market indices. PMID:27293423

Financial Time Series Prediction Using Elman Recurrent Random Neural Networks.

PubMed

Wang, Jie; Wang, Jun; Fang, Wen; Niu, Hongli

2016-01-01

In recent years, financial market dynamics forecasting has been a focus of economic research. To predict the price indices of stock markets, we developed an architecture which combined Elman recurrent neural networks with stochastic time effective function. By analyzing the proposed model with the linear regression, complexity invariant distance (CID), and multiscale CID (MCID) analysis methods and taking the model compared with different models such as the backpropagation neural network (BPNN), the stochastic time effective neural network (STNN), and the Elman recurrent neural network (ERNN), the empirical results show that the proposed neural network displays the best performance among these neural networks in financial time series forecasting. Further, the empirical research is performed in testing the predictive effects of SSE, TWSE, KOSPI, and Nikkei225 with the established model, and the corresponding statistical comparisons of the above market indices are also exhibited. The experimental results show that this approach gives good performance in predicting the values from the stock market indices.

Modeling Belt-Servomechanism by Chebyshev Functional Recurrent Neuro-Fuzzy Network

NASA Astrophysics Data System (ADS)

Huang, Yuan-Ruey; Kang, Yuan; Chu, Ming-Hui; Chang, Yeon-Pun

A novel Chebyshev functional recurrent neuro-fuzzy (CFRNF) network is developed from a combination of the Takagi-Sugeno-Kang (TSK) fuzzy model and the Chebyshev recurrent neural network (CRNN). The CFRNF network can emulate the nonlinear dynamics of a servomechanism system. The system nonlinearity is addressed by enhancing the input dimensions of the consequent parts in the fuzzy rules due to functional expansion of a Chebyshev polynomial. The back propagation algorithm is used to adjust the parameters of the antecedent membership functions as well as those of consequent functions. To verify the performance of the proposed CFRNF, the experiment of the belt servomechanism is presented in this paper. Both of identification methods of adaptive neural fuzzy inference system (ANFIS) and recurrent neural network (RNN) are also studied for modeling of the belt servomechanism. The analysis and comparison results indicate that CFRNF makes identification of complex nonlinear dynamic systems easier. It is verified that the accuracy and convergence of the CFRNF are superior to those of ANFIS and RNN by the identification results of a belt servomechanism.

Cross over of recurrence networks to random graphs and random geometric graphs

NASA Astrophysics Data System (ADS)

Jacob, Rinku; Harikrishnan, K. P.; Misra, R.; Ambika, G.

2017-02-01

Recurrence networks are complex networks constructed from the time series of chaotic dynamical systems where the connection between two nodes is limited by the recurrence threshold. This condition makes the topology of every recurrence network unique with the degree distribution determined by the probability density variations of the representative attractor from which it is constructed. Here we numerically investigate the properties of recurrence networks from standard low-dimensional chaotic attractors using some basic network measures and show how the recurrence networks are different from random and scale-free networks. In particular, we show that all recurrence networks can cross over to random geometric graphs by adding sufficient amount of noise to the time series and into the classical random graphs by increasing the range of interaction to the system size. We also highlight the effectiveness of a combined plot of characteristic path length and clustering coefficient in capturing the small changes in the network characteristics.

Persistence and storage of activity patterns in spiking recurrent cortical networks: modulation of sigmoid signals by after-hyperpolarization currents and acetylcholine

PubMed Central

Palma, Jesse; Grossberg, Stephen; Versace, Massimiliano

2012-01-01

Many cortical networks contain recurrent architectures that transform input patterns before storing them in short-term memory (STM). Theorems in the 1970's showed how feedback signal functions in rate-based recurrent on-center off-surround networks control this process. A sigmoid signal function induces a quenching threshold below which inputs are suppressed as noise and above which they are contrast-enhanced before pattern storage. This article describes how changes in feedback signaling, neuromodulation, and recurrent connectivity may alter pattern processing in recurrent on-center off-surround networks of spiking neurons. In spiking neurons, fast, medium, and slow after-hyperpolarization (AHP) currents control sigmoid signal threshold and slope. Modulation of AHP currents by acetylcholine (ACh) can change sigmoid shape and, with it, network dynamics. For example, decreasing signal function threshold and increasing slope can lengthen the persistence of a partially contrast-enhanced pattern, increase the number of active cells stored in STM, or, if connectivity is distance-dependent, cause cell activities to cluster. These results clarify how cholinergic modulation by the basal forebrain may alter the vigilance of category learning circuits, and thus their sensitivity to predictive mismatches, thereby controlling whether learned categories code concrete or abstract features, as predicted by Adaptive Resonance Theory. The analysis includes global, distance-dependent, and interneuron-mediated circuits. With an appropriate degree of recurrent excitation and inhibition, spiking networks maintain a partially contrast-enhanced pattern for 800 ms or longer after stimuli offset, then resolve to no stored pattern, or to winner-take-all (WTA) stored patterns with one or multiple winners. Strengthening inhibition prolongs a partially contrast-enhanced pattern by slowing the transition to stability, while strengthening excitation causes more winners when the network stabilizes. PMID:22754524

A recurrent neural network for nonlinear optimization with a continuously differentiable objective function and bound constraints.

PubMed

Liang, X B; Wang, J

2000-01-01

This paper presents a continuous-time recurrent neural-network model for nonlinear optimization with any continuously differentiable objective function and bound constraints. Quadratic optimization with bound constraints is a special problem which can be solved by the recurrent neural network. The proposed recurrent neural network has the following characteristics. 1) It is regular in the sense that any optimum of the objective function with bound constraints is also an equilibrium point of the neural network. If the objective function to be minimized is convex, then the recurrent neural network is complete in the sense that the set of optima of the function with bound constraints coincides with the set of equilibria of the neural network. 2) The recurrent neural network is primal and quasiconvergent in the sense that its trajectory cannot escape from the feasible region and will converge to the set of equilibria of the neural network for any initial point in the feasible bound region. 3) The recurrent neural network has an attractivity property in the sense that its trajectory will eventually converge to the feasible region for any initial states even at outside of the bounded feasible region. 4) For minimizing any strictly convex quadratic objective function subject to bound constraints, the recurrent neural network is globally exponentially stable for almost any positive network parameters. Simulation results are given to demonstrate the convergence and performance of the proposed recurrent neural network for nonlinear optimization with bound constraints.

A recurrent neural network for solving bilevel linear programming problem.

PubMed

He, Xing; Li, Chuandong; Huang, Tingwen; Li, Chaojie; Huang, Junjian

2014-04-01

In this brief, based on the method of penalty functions, a recurrent neural network (NN) modeled by means of a differential inclusion is proposed for solving the bilevel linear programming problem (BLPP). Compared with the existing NNs for BLPP, the model has the least number of state variables and simple structure. Using nonsmooth analysis, the theory of differential inclusions, and Lyapunov-like method, the equilibrium point sequence of the proposed NNs can approximately converge to an optimal solution of BLPP under certain conditions. Finally, the numerical simulations of a supply chain distribution model have shown excellent performance of the proposed recurrent NNs.

Low-dimensional recurrent neural network-based Kalman filter for speech enhancement.

PubMed

Xia, Youshen; Wang, Jun

2015-07-01

This paper proposes a new recurrent neural network-based Kalman filter for speech enhancement, based on a noise-constrained least squares estimate. The parameters of speech signal modeled as autoregressive process are first estimated by using the proposed recurrent neural network and the speech signal is then recovered from Kalman filtering. The proposed recurrent neural network is globally asymptomatically stable to the noise-constrained estimate. Because the noise-constrained estimate has a robust performance against non-Gaussian noise, the proposed recurrent neural network-based speech enhancement algorithm can minimize the estimation error of Kalman filter parameters in non-Gaussian noise. Furthermore, having a low-dimensional model feature, the proposed neural network-based speech enhancement algorithm has a much faster speed than two existing recurrent neural networks-based speech enhancement algorithms. Simulation results show that the proposed recurrent neural network-based speech enhancement algorithm can produce a good performance with fast computation and noise reduction. Copyright © 2015 Elsevier Ltd. All rights reserved.

Dynamic stability analysis of fractional order leaky integrator echo state neural networks

NASA Astrophysics Data System (ADS)

Pahnehkolaei, Seyed Mehdi Abedi; Alfi, Alireza; Tenreiro Machado, J. A.

2017-06-01

The Leaky integrator echo state neural network (Leaky-ESN) is an improved model of the recurrent neural network (RNN) and adopts an interconnected recurrent grid of processing neurons. This paper presents a new proof for the convergence of a Lyapunov candidate function to zero when time tends to infinity by means of the Caputo fractional derivative with order lying in the range (0, 1). The stability of Fractional-Order Leaky-ESN (FO Leaky-ESN) is then analyzed, and the existence, uniqueness and stability of the equilibrium point are provided. A numerical example demonstrates the feasibility of the proposed method.

Distribution of Orientation Selectivity in Recurrent Networks of Spiking Neurons with Different Random Topologies

PubMed Central

Sadeh, Sadra; Rotter, Stefan

2014-01-01

Neurons in the primary visual cortex are more or less selective for the orientation of a light bar used for stimulation. A broad distribution of individual grades of orientation selectivity has in fact been reported in all species. A possible reason for emergence of broad distributions is the recurrent network within which the stimulus is being processed. Here we compute the distribution of orientation selectivity in randomly connected model networks that are equipped with different spatial patterns of connectivity. We show that, for a wide variety of connectivity patterns, a linear theory based on firing rates accurately approximates the outcome of direct numerical simulations of networks of spiking neurons. Distance dependent connectivity in networks with a more biologically realistic structure does not compromise our linear analysis, as long as the linearized dynamics, and hence the uniform asynchronous irregular activity state, remain stable. We conclude that linear mechanisms of stimulus processing are indeed responsible for the emergence of orientation selectivity and its distribution in recurrent networks with functionally heterogeneous synaptic connectivity. PMID:25469704

Encoding Time in Feedforward Trajectories of a Recurrent Neural Network Model.

PubMed

Hardy, N F; Buonomano, Dean V

2018-02-01

Brain activity evolves through time, creating trajectories of activity that underlie sensorimotor processing, behavior, and learning and memory. Therefore, understanding the temporal nature of neural dynamics is essential to understanding brain function and behavior. In vivo studies have demonstrated that sequential transient activation of neurons can encode time. However, it remains unclear whether these patterns emerge from feedforward network architectures or from recurrent networks and, furthermore, what role network structure plays in timing. We address these issues using a recurrent neural network (RNN) model with distinct populations of excitatory and inhibitory units. Consistent with experimental data, a single RNN could autonomously produce multiple functionally feedforward trajectories, thus potentially encoding multiple timed motor patterns lasting up to several seconds. Importantly, the model accounted for Weber's law, a hallmark of timing behavior. Analysis of network connectivity revealed that efficiency-a measure of network interconnectedness-decreased as the number of stored trajectories increased. Additionally, the balance of excitation (E) and inhibition (I) shifted toward excitation during each unit's activation time, generating the prediction that observed sequential activity relies on dynamic control of the E/I balance. Our results establish for the first time that the same RNN can generate multiple functionally feedforward patterns of activity as a result of dynamic shifts in the E/I balance imposed by the connectome of the RNN. We conclude that recurrent network architectures account for sequential neural activity, as well as for a fundamental signature of timing behavior: Weber's law.

A mathematical analysis of the effects of Hebbian learning rules on the dynamics and structure of discrete-time random recurrent neural networks.

PubMed

Siri, Benoît; Berry, Hugues; Cessac, Bruno; Delord, Bruno; Quoy, Mathias

2008-12-01

We present a mathematical analysis of the effects of Hebbian learning in random recurrent neural networks, with a generic Hebbian learning rule, including passive forgetting and different timescales, for neuronal activity and learning dynamics. Previous numerical work has reported that Hebbian learning drives the system from chaos to a steady state through a sequence of bifurcations. Here, we interpret these results mathematically and show that these effects, involving a complex coupling between neuronal dynamics and synaptic graph structure, can be analyzed using Jacobian matrices, which introduce both a structural and a dynamical point of view on neural network evolution. Furthermore, we show that sensitivity to a learned pattern is maximal when the largest Lyapunov exponent is close to 0. We discuss how neural networks may take advantage of this regime of high functional interest.

Dynamics of intracranial electroencephalographic recordings from epilepsy patients using univariate and bivariate recurrence networks.

PubMed

Subramaniyam, Narayan Puthanmadam; Hyttinen, Jari

2015-02-01

Recently Andrezejak et al. combined the randomness and nonlinear independence test with iterative amplitude adjusted Fourier transform (iAAFT) surrogates to distinguish between the dynamics of seizure-free intracranial electroencephalographic (EEG) signals recorded from epileptogenic (focal) and nonepileptogenic (nonfocal) brain areas of epileptic patients. However, stationarity is a part of the null hypothesis for iAAFT surrogates and thus nonstationarity can violate the null hypothesis. In this work we first propose the application of the randomness and nonlinear independence test based on recurrence network measures to distinguish between the dynamics of focal and nonfocal EEG signals. Furthermore, we combine these tests with both iAAFT and truncated Fourier transform (TFT) surrogate methods, which also preserves the nonstationarity of the original data in the surrogates along with its linear structure. Our results indicate that focal EEG signals exhibit an increased degree of structural complexity and interdependency compared to nonfocal EEG signals. In general, we find higher rejections for randomness and nonlinear independence tests for focal EEG signals compared to nonfocal EEG signals. In particular, the univariate recurrence network measures, the average clustering coefficient C and assortativity R, and the bivariate recurrence network measure, the average cross-clustering coefficient C(cross), can successfully distinguish between the focal and nonfocal EEG signals, even when the analysis is restricted to nonstationary signals, irrespective of the type of surrogates used. On the other hand, we find that the univariate recurrence network measures, the average path length L, and the average betweenness centrality BC fail to distinguish between the focal and nonfocal EEG signals when iAAFT surrogates are used. However, these two measures can distinguish between focal and nonfocal EEG signals when TFT surrogates are used for nonstationary signals. We also report an improvement in the performance of nonlinear prediction error N and nonlinear interdependence measure L used by Andrezejak et al., when TFT surrogates are used for nonstationary EEG signals. We also find that the outcome of the nonlinear independence test based on the average cross-clustering coefficient C(cross) is independent of the outcome of the randomness test based on the average clustering coefficient C. Thus, the univariate and bivariate recurrence network measures provide independent information regarding the dynamics of the focal and nonfocal EEG signals. In conclusion, recurrence network analysis combined with nonstationary surrogates can be applied to derive reliable biomarkers to distinguish between epileptogenic and nonepileptogenic brain areas using EEG signals.

Dynamics of intracranial electroencephalographic recordings from epilepsy patients using univariate and bivariate recurrence networks

NASA Astrophysics Data System (ADS)

Subramaniyam, Narayan Puthanmadam; Hyttinen, Jari

2015-02-01

Recently Andrezejak et al. combined the randomness and nonlinear independence test with iterative amplitude adjusted Fourier transform (iAAFT) surrogates to distinguish between the dynamics of seizure-free intracranial electroencephalographic (EEG) signals recorded from epileptogenic (focal) and nonepileptogenic (nonfocal) brain areas of epileptic patients. However, stationarity is a part of the null hypothesis for iAAFT surrogates and thus nonstationarity can violate the null hypothesis. In this work we first propose the application of the randomness and nonlinear independence test based on recurrence network measures to distinguish between the dynamics of focal and nonfocal EEG signals. Furthermore, we combine these tests with both iAAFT and truncated Fourier transform (TFT) surrogate methods, which also preserves the nonstationarity of the original data in the surrogates along with its linear structure. Our results indicate that focal EEG signals exhibit an increased degree of structural complexity and interdependency compared to nonfocal EEG signals. In general, we find higher rejections for randomness and nonlinear independence tests for focal EEG signals compared to nonfocal EEG signals. In particular, the univariate recurrence network measures, the average clustering coefficient C and assortativity R , and the bivariate recurrence network measure, the average cross-clustering coefficient Ccross, can successfully distinguish between the focal and nonfocal EEG signals, even when the analysis is restricted to nonstationary signals, irrespective of the type of surrogates used. On the other hand, we find that the univariate recurrence network measures, the average path length L , and the average betweenness centrality BC fail to distinguish between the focal and nonfocal EEG signals when iAAFT surrogates are used. However, these two measures can distinguish between focal and nonfocal EEG signals when TFT surrogates are used for nonstationary signals. We also report an improvement in the performance of nonlinear prediction error N and nonlinear interdependence measure L used by Andrezejak et al., when TFT surrogates are used for nonstationary EEG signals. We also find that the outcome of the nonlinear independence test based on the average cross-clustering coefficient Ccross is independent of the outcome of the randomness test based on the average clustering coefficient C . Thus, the univariate and bivariate recurrence network measures provide independent information regarding the dynamics of the focal and nonfocal EEG signals. In conclusion, recurrence network analysis combined with nonstationary surrogates can be applied to derive reliable biomarkers to distinguish between epileptogenic and nonepileptogenic brain areas using EEG signals.

Deep Neural Network for Structural Prediction and Lane Detection in Traffic Scene.

PubMed

Li, Jun; Mei, Xue; Prokhorov, Danil; Tao, Dacheng

2017-03-01

Hierarchical neural networks have been shown to be effective in learning representative image features and recognizing object classes. However, most existing networks combine the low/middle level cues for classification without accounting for any spatial structures. For applications such as understanding a scene, how the visual cues are spatially distributed in an image becomes essential for successful analysis. This paper extends the framework of deep neural networks by accounting for the structural cues in the visual signals. In particular, two kinds of neural networks have been proposed. First, we develop a multitask deep convolutional network, which simultaneously detects the presence of the target and the geometric attributes (location and orientation) of the target with respect to the region of interest. Second, a recurrent neuron layer is adopted for structured visual detection. The recurrent neurons can deal with the spatial distribution of visible cues belonging to an object whose shape or structure is difficult to explicitly define. Both the networks are demonstrated by the practical task of detecting lane boundaries in traffic scenes. The multitask convolutional neural network provides auxiliary geometric information to help the subsequent modeling of the given lane structures. The recurrent neural network automatically detects lane boundaries, including those areas containing no marks, without any explicit prior knowledge or secondary modeling.

Evaluation of selected recurrence measures in discriminating pre-ictal and inter-ictal periods from epileptic EEG data

NASA Astrophysics Data System (ADS)

Ngamga, Eulalie Joelle; Bialonski, Stephan; Marwan, Norbert; Kurths, Jürgen; Geier, Christian; Lehnertz, Klaus

2016-04-01

We investigate the suitability of selected measures of complexity based on recurrence quantification analysis and recurrence networks for an identification of pre-seizure states in multi-day, multi-channel, invasive electroencephalographic recordings from five epilepsy patients. We employ several statistical techniques to avoid spurious findings due to various influencing factors and due to multiple comparisons and observe precursory structures in three patients. Our findings indicate a high congruence among measures in identifying seizure precursors and emphasize the current notion of seizure generation in large-scale epileptic networks. A final judgment of the suitability for field studies, however, requires evaluation on a larger database.

A class of finite-time dual neural networks for solving quadratic programming problems and its k-winners-take-all application.

PubMed

Li, Shuai; Li, Yangming; Wang, Zheng

2013-03-01

This paper presents a class of recurrent neural networks to solve quadratic programming problems. Different from most existing recurrent neural networks for solving quadratic programming problems, the proposed neural network model converges in finite time and the activation function is not required to be a hard-limiting function for finite convergence time. The stability, finite-time convergence property and the optimality of the proposed neural network for solving the original quadratic programming problem are proven in theory. Extensive simulations are performed to evaluate the performance of the neural network with different parameters. In addition, the proposed neural network is applied to solving the k-winner-take-all (k-WTA) problem. Both theoretical analysis and numerical simulations validate the effectiveness of our method for solving the k-WTA problem. Copyright © 2012 Elsevier Ltd. All rights reserved.

Structure simulation into a lamellar supramolecular network and calculation of the metal ions/ligands ratio

PubMed Central

2012-01-01

Background Research interest in phosphonates metal organic frameworks (MOF) has increased extremely in the last two decades, because of theirs fascinating and complex topology and structural flexibility. In this paper we present a mathematical model for ligand/metal ion ratio of an octahedral (Oh) network of cobalt vinylphosphonate (Co(vP)·H2O). Results A recurrent relationship of the ratio between the number of ligands and the number of metal ions in a lamellar octahedral (Oh) network Co(vP)·H2O, has been deducted by building the 3D network step by step using HyperChem 7.52 package. The mathematical relationship has been validated using X ray analysis, experimental thermogravimetric and elemental analysis data. Conclusions Based on deducted recurrence relationship, we can conclude prior to perform X ray analysis, that in the case of a thermogravimetric analysis pointing a ratio between the number of metal ions and ligands number around 1, the 3D network will have a central metal ion that corresponds to a single ligand. This relation is valid for every type of supramolecular network with divalent metal central ion Oh coordinated and bring valuable information with low effort and cost. PMID:22932493

A novel joint-processing adaptive nonlinear equalizer using a modular recurrent neural network for chaotic communication systems.

PubMed

Zhao, Haiquan; Zeng, Xiangping; Zhang, Jiashu; Liu, Yangguang; Wang, Xiaomin; Li, Tianrui

2011-01-01

To eliminate nonlinear channel distortion in chaotic communication systems, a novel joint-processing adaptive nonlinear equalizer based on a pipelined recurrent neural network (JPRNN) is proposed, using a modified real-time recurrent learning (RTRL) algorithm. Furthermore, an adaptive amplitude RTRL algorithm is adopted to overcome the deteriorating effect introduced by the nesting process. Computer simulations illustrate that the proposed equalizer outperforms the pipelined recurrent neural network (PRNN) and recurrent neural network (RNN) equalizers. Copyright © 2010 Elsevier Ltd. All rights reserved.

Embedding recurrent neural networks into predator-prey models.

PubMed

Moreau, Yves; Louiès, Stephane; Vandewalle, Joos; Brenig, Leon

1999-03-01

We study changes of coordinates that allow the embedding of ordinary differential equations describing continuous-time recurrent neural networks into differential equations describing predator-prey models-also called Lotka-Volterra systems. We transform the equations for the neural network first into quasi-monomial form (Brenig, L. (1988). Complete factorization and analytic solutions of generalized Lotka-Volterra equations. Physics Letters A, 133(7-8), 378-382), where we express the vector field of the dynamical system as a linear combination of products of powers of the variables. In practice, this transformation is possible only if the activation function is the hyperbolic tangent or the logistic sigmoid. From this quasi-monomial form, we can directly transform the system further into Lotka-Volterra equations. The resulting Lotka-Volterra system is of higher dimension than the original system, but the behavior of its first variables is equivalent to the behavior of the original neural network. We expect that this transformation will permit the application of existing techniques for the analysis of Lotka-Volterra systems to recurrent neural networks. Furthermore, our results show that Lotka-Volterra systems are universal approximators of dynamical systems, just as are continuous-time neural networks.

Delay-slope-dependent stability results of recurrent neural networks.

PubMed

Li, Tao; Zheng, Wei Xing; Lin, Chong

2011-12-01

By using the fact that the neuron activation functions are sector bounded and nondecreasing, this brief presents a new method, named the delay-slope-dependent method, for stability analysis of a class of recurrent neural networks with time-varying delays. This method includes more information on the slope of neuron activation functions and fewer matrix variables in the constructed Lyapunov-Krasovskii functional. Then some improved delay-dependent stability criteria with less computational burden and conservatism are obtained. Numerical examples are given to illustrate the effectiveness and the benefits of the proposed method.

Bankruptcy prediction based on financial ratios using Jordan Recurrent Neural Networks: a case study in Polish companies

NASA Astrophysics Data System (ADS)

Hardinata, Lingga; Warsito, Budi; Suparti

2018-05-01

Complexity of bankruptcy causes the accurate models of bankruptcy prediction difficult to be achieved. Various prediction models have been developed to improve the accuracy of bankruptcy predictions. Machine learning has been widely used to predict because of its adaptive capabilities. Artificial Neural Networks (ANN) is one of machine learning which proved able to complete inference tasks such as prediction and classification especially in data mining. In this paper, we propose the implementation of Jordan Recurrent Neural Networks (JRNN) to classify and predict corporate bankruptcy based on financial ratios. Feedback interconnection in JRNN enable to make the network keep important information well allowing the network to work more effectively. The result analysis showed that JRNN works very well in bankruptcy prediction with average success rate of 81.3785%.

Character recognition from trajectory by recurrent spiking neural networks.

PubMed

Jiangrong Shen; Kang Lin; Yueming Wang; Gang Pan

2017-07-01

Spiking neural networks are biologically plausible and power-efficient on neuromorphic hardware, while recurrent neural networks have been proven to be efficient on time series data. However, how to use the recurrent property to improve the performance of spiking neural networks is still a problem. This paper proposes a recurrent spiking neural network for character recognition using trajectories. In the network, a new encoding method is designed, in which varying time ranges of input streams are used in different recurrent layers. This is able to improve the generalization ability of our model compared with general encoding methods. The experiments are conducted on four groups of the character data set from University of Edinburgh. The results show that our method can achieve a higher average recognition accuracy than existing methods.

MiRNA and TF co-regulatory network analysis for the pathology and recurrence of myocardial infarction.

PubMed

Lin, Ying; Sibanda, Vusumuzi Leroy; Zhang, Hong-Mei; Hu, Hui; Liu, Hui; Guo, An-Yuan

2015-04-13

Myocardial infarction (MI) is a leading cause of death in the world and many genes are involved in it. Transcription factor (TFs) and microRNAs (miRNAs) are key regulators of gene expression. We hypothesized that miRNAs and TFs might play combinatory regulatory roles in MI. After collecting MI candidate genes and miRNAs from various resources, we constructed a comprehensive MI-specific miRNA-TF co-regulatory network by integrating predicted and experimentally validated TF and miRNA targets. We found some hub nodes (e.g. miR-16 and miR-26) in this network are important regulators, and the network can be severed as a bridge to interpret the associations of previous results, which is shown by the case of miR-29 in this study. We also constructed a regulatory network for MI recurrence and found several important genes (e.g. DAB2, BMP6, miR-320 and miR-103), the abnormal expressions of which may be potential regulatory mechanisms and markers of MI recurrence. At last we proposed a cellular model to discuss major TF and miRNA regulators with signaling pathways in MI. This study provides more details on gene expression regulation and regulators involved in MI progression and recurrence. It also linked up and interpreted many previous results.

The geometry of chaotic dynamics — a complex network perspective

NASA Astrophysics Data System (ADS)

Donner, R. V.; Heitzig, J.; Donges, J. F.; Zou, Y.; Marwan, N.; Kurths, J.

2011-12-01

Recently, several complex network approaches to time series analysis have been developed and applied to study a wide range of model systems as well as real-world data, e.g., geophysical or financial time series. Among these techniques, recurrence-based concepts and prominently ɛ-recurrence networks, most faithfully represent the geometrical fine structure of the attractors underlying chaotic (and less interestingly non-chaotic) time series. In this paper we demonstrate that the well known graph theoretical properties local clustering coefficient and global (network) transitivity can meaningfully be exploited to define two new local and two new global measures of dimension in phase space: local upper and lower clustering dimension as well as global upper and lower transitivity dimension. Rigorous analytical as well as numerical results for self-similar sets and simple chaotic model systems suggest that these measures are well-behaved in most non-pathological situations and that they can be estimated reasonably well using ɛ-recurrence networks constructed from relatively short time series. Moreover, we study the relationship between clustering and transitivity dimensions on the one hand, and traditional measures like pointwise dimension or local Lyapunov dimension on the other hand. We also provide further evidence that the local clustering coefficients, or equivalently the local clustering dimensions, are useful for identifying unstable periodic orbits and other dynamically invariant objects from time series. Our results demonstrate that ɛ-recurrence networks exhibit an important link between dynamical systems and graph theory.

Identification of cyclin B1 and Sec62 as biomarkers for recurrence in patients with HBV-related hepatocellular carcinoma after surgical resection.

PubMed

Weng, Li; Du, Juan; Zhou, Qinghui; Cheng, Binbin; Li, Jun; Zhang, Denghai; Ling, Changquan

2012-06-08

Hepatocellular carcinoma (HCC) is the fifth most common cancer worldwide. Frequent tumor recurrence after surgery is related to its poor prognosis. Although gene expression signatures have been associated with outcome, the molecular basis of HCC recurrence is not fully understood, and there is no method to predict recurrence using peripheral blood mononuclear cells (PBMCs), which can be easily obtained for recurrence prediction in the clinical setting. According to the microarray analysis results, we constructed a co-expression network using the k-core algorithm to determine which genes play pivotal roles in the recurrence of HCC associated with the hepatitis B virus (HBV) infection. Furthermore, we evaluated the mRNA and protein expressions in the PBMCs from 80 patients with or without recurrence and 30 healthy subjects. The stability of the signatures was determined in HCC tissues from the same 80 patients. Data analysis included ROC analysis, correlation analysis, log-lank tests, and Cox modeling to identify independent predictors of tumor recurrence. The tumor-associated proteins cyclin B1, Sec62, and Birc3 were highly expressed in a subset of samples of recurrent HCC; cyclin B1, Sec62, and Birc3 positivity was observed in 80%, 65.7%, and 54.2% of the samples, respectively. The Kaplan-Meier analysis revealed that high expression levels of these proteins was associated with significantly reduced recurrence-free survival. Cox proportional hazards model analysis revealed that cyclin B1 (hazard ratio [HR], 4.762; p = 0.002) and Sec62 (HR, 2.674; p = 0.018) were independent predictors of HCC recurrence. These results revealed that cyclin B1 and Sec62 may be candidate biomarkers and potential therapeutic targets for HBV-related HCC recurrence after surgery.

Circular RNA In Invasive and Recurrent Clinical Nonfunctioning Pituitary Adenomas: Expression Profiles and Bioinformatic Analysis.

PubMed

Wang, Jianpeng; Wang, Dong; Wan, Dehong; Ma, Qingxia; Liu, Qian; Li, Jiye; Li, Zhaojian; Gao, Yang; Jiang, Guohui; Ma, Leina; Liu, Jia; Li, Chuzhong

2018-06-14

The invasion and recurrence of clinical nonfunctioning pituitary adenomas (NFA) often lead to surgical treatment failure. Circular RNAs (circRNAs) are a novel class of RNAs whose 3' and 5' ends are joined together and have been shown to play important roles in cancer development. Up to now, the roles of circRNAs remain unclear in invasive and recurrent NFA. We detected and summarized the circRNA expression pattern in 75 NFA tissues from 10 non-invasive cases and 65 invasive cases and 9 pairs NFA tumor tissues from 9 recurrent cases by circRNA microarrays. Accordingly, functional enrichment analysis and pathway analysis were performed and circRNA-microRNA(miRNA) network were generated by bioinformatic analysis tools. 5 new invasive NFA samples and 5 non-invasive NFA samples were collected to measure the microarray results. 570 dysregulated circRNAs (Invasive Tumor vs. Non-invasive Tumor) and 10 up-regulated circRNAs (Recurrent tumor Tissue vs. First surgery tumor Tissue) were identified based on the situation (FC>2, P<0.05). The parental genes of the dysregulated circRNAs in the comparison between invasion tumor and non-invasion tumor were found to be enriched in some cell adhesion signaling pathways such as Focal adhesion, Hippo signaling pathway, PI3K-Akt signaling pathway, and Adherens junction. The circRNA-miRNA network showed that the dysregulated circRNA may function as miRNA sponges. This is the first study to conduct and comprehensively analyze the circRNA expression profile in invasive and recurrent NFA. Our finding will provide evidence for the significance of circRNAs in NFA diagnosis, prognosis and clinical treatment. Copyright © 2018 Elsevier Inc. All rights reserved.

Recurrence of IgA nephropathy after kidney transplantation in steroid continuation versus early steroid-withdrawal regimens: a retrospective analysis of the UNOS/OPTN database.

PubMed

Leeaphorn, Napat; Garg, Neetika; Khankin, Eliyahu V; Cardarelli, Francesca; Pavlakis, Martha

2018-02-01

In the past 20 years, there has been an increase in use of steroid-withdrawal regimens in kidney transplantation. However, steroid withdrawal may be associated with an increased risk of recurrent IgA nephropathy (IgAN). Using United Network of (Organ Sharing/Organ Procurement and Transplantation Network) UNOS/OPTN data, we analyzed adult patients with end-stage renal disease (ESRD) due to IgAN who received their first kidney transplant between 2000 and 2014. For the primary outcome, we used a competing risk analysis to compare the cumulative incidence of graft loss due to IgAN recurrence between early steroid-withdrawal (ESW) and steroid continuation groups. The secondary outcomes were patient survival and death-censored graft survival (DCGS). A total of 9690 recipients were included (2831 in ESW group and 6859 in steroid continuation group). In total, 1238 recipients experienced graft loss, of which 191 (15.43%) were due to IgAN recurrence. In multivariable analysis, steroid use was associated with a decreased risk of recurrence (subdistribution hazard ratio 0.666, 95% CI 0.482-0.921; P = 0.014). Patient survival and DCGS were not different between the two groups. In the USA, ESW in transplant for ESRD due to IgAN is associated with a higher risk of graft loss due to disease recurrence. Future prospective studies are warranted to further address which patients with IgAN would benefit from steroid continuation. © 2017 Steunstichting ESOT.

Recurrent Convolutional Neural Networks: A Better Model of Biological Object Recognition.

PubMed

Spoerer, Courtney J; McClure, Patrick; Kriegeskorte, Nikolaus

2017-01-01

Feedforward neural networks provide the dominant model of how the brain performs visual object recognition. However, these networks lack the lateral and feedback connections, and the resulting recurrent neuronal dynamics, of the ventral visual pathway in the human and non-human primate brain. Here we investigate recurrent convolutional neural networks with bottom-up (B), lateral (L), and top-down (T) connections. Combining these types of connections yields four architectures (B, BT, BL, and BLT), which we systematically test and compare. We hypothesized that recurrent dynamics might improve recognition performance in the challenging scenario of partial occlusion. We introduce two novel occluded object recognition tasks to test the efficacy of the models, digit clutter (where multiple target digits occlude one another) and digit debris (where target digits are occluded by digit fragments). We find that recurrent neural networks outperform feedforward control models (approximately matched in parametric complexity) at recognizing objects, both in the absence of occlusion and in all occlusion conditions. Recurrent networks were also found to be more robust to the inclusion of additive Gaussian noise. Recurrent neural networks are better in two respects: (1) they are more neurobiologically realistic than their feedforward counterparts; (2) they are better in terms of their ability to recognize objects, especially under challenging conditions. This work shows that computer vision can benefit from using recurrent convolutional architectures and suggests that the ubiquitous recurrent connections in biological brains are essential for task performance.

Nonlinear recurrent neural networks for finite-time solution of general time-varying linear matrix equations.

PubMed

Xiao, Lin; Liao, Bolin; Li, Shuai; Chen, Ke

2018-02-01

In order to solve general time-varying linear matrix equations (LMEs) more efficiently, this paper proposes two nonlinear recurrent neural networks based on two nonlinear activation functions. According to Lyapunov theory, such two nonlinear recurrent neural networks are proved to be convergent within finite-time. Besides, by solving differential equation, the upper bounds of the finite convergence time are determined analytically. Compared with existing recurrent neural networks, the proposed two nonlinear recurrent neural networks have a better convergence property (i.e., the upper bound is lower), and thus the accurate solutions of general time-varying LMEs can be obtained with less time. At last, various different situations have been considered by setting different coefficient matrices of general time-varying LMEs and a great variety of computer simulations (including the application to robot manipulators) have been conducted to validate the better finite-time convergence of the proposed two nonlinear recurrent neural networks. Copyright © 2017 Elsevier Ltd. All rights reserved.

Solving differential equations with unknown constitutive relations as recurrent neural networks

DOE Office of Scientific and Technical Information (OSTI.GOV)

Hagge, Tobias J.; Stinis, Panagiotis; Yeung, Enoch H.

We solve a system of ordinary differential equations with an unknown functional form of a sink (reaction rate) term. We assume that the measurements (time series) of state variables are partially available, and use a recurrent neural network to “learn” the reaction rate from this data. This is achieved by including discretized ordinary differential equations as part of a recurrent neural network training problem. We extend TensorFlow’s recurrent neural network architecture to create a simple but scalable and effective solver for the unknown functions, and apply it to a fedbatch bioreactor simulation problem. Use of techniques from recent deep learningmore » literature enables training of functions with behavior manifesting over thousands of time steps. Our networks are structurally similar to recurrent neural networks, but differ in purpose, and require modified training strategies.« less

Multiscale recurrence analysis of spatio-temporal data

NASA Astrophysics Data System (ADS)

Riedl, M.; Marwan, N.; Kurths, J.

2015-12-01

The description and analysis of spatio-temporal dynamics is a crucial task in many scientific disciplines. In this work, we propose a method which uses the mapogram as a similarity measure between spatially distributed data instances at different time points. The resulting similarity values of the pairwise comparison are used to construct a recurrence plot in order to benefit from established tools of recurrence quantification analysis and recurrence network analysis. In contrast to other recurrence tools for this purpose, the mapogram approach allows the specific focus on different spatial scales that can be used in a multi-scale analysis of spatio-temporal dynamics. We illustrate this approach by application on mixed dynamics, such as traveling parallel wave fronts with additive noise, as well as more complicate examples, pseudo-random numbers and coupled map lattices with a semi-logistic mapping rule. Especially the complicate examples show the usefulness of the multi-scale consideration in order to take spatial pattern of different scales and with different rhythms into account. So, this mapogram approach promises new insights in problems of climatology, ecology, or medicine.

Multiscale recurrence analysis of spatio-temporal data.

PubMed

Riedl, M; Marwan, N; Kurths, J

2015-12-01

The description and analysis of spatio-temporal dynamics is a crucial task in many scientific disciplines. In this work, we propose a method which uses the mapogram as a similarity measure between spatially distributed data instances at different time points. The resulting similarity values of the pairwise comparison are used to construct a recurrence plot in order to benefit from established tools of recurrence quantification analysis and recurrence network analysis. In contrast to other recurrence tools for this purpose, the mapogram approach allows the specific focus on different spatial scales that can be used in a multi-scale analysis of spatio-temporal dynamics. We illustrate this approach by application on mixed dynamics, such as traveling parallel wave fronts with additive noise, as well as more complicate examples, pseudo-random numbers and coupled map lattices with a semi-logistic mapping rule. Especially the complicate examples show the usefulness of the multi-scale consideration in order to take spatial pattern of different scales and with different rhythms into account. So, this mapogram approach promises new insights in problems of climatology, ecology, or medicine.

Collective stochastic coherence in recurrent neuronal networks

NASA Astrophysics Data System (ADS)

Sancristóbal, Belén; Rebollo, Beatriz; Boada, Pol; Sanchez-Vives, Maria V.; Garcia-Ojalvo, Jordi

2016-09-01

Recurrent networks of dynamic elements frequently exhibit emergent collective oscillations, which can show substantial regularity even when the individual elements are considerably noisy. How noise-induced dynamics at the local level coexists with regular oscillations at the global level is still unclear. Here we show that a combination of stochastic recurrence-based initiation with deterministic refractoriness in an excitable network can reconcile these two features, leading to maximum collective coherence for an intermediate noise level. We report this behaviour in the slow oscillation regime exhibited by a cerebral cortex network under dynamical conditions resembling slow-wave sleep and anaesthesia. Computational analysis of a biologically realistic network model reveals that an intermediate level of background noise leads to quasi-regular dynamics. We verify this prediction experimentally in cortical slices subject to varying amounts of extracellular potassium, which modulates neuronal excitability and thus synaptic noise. The model also predicts that this effectively regular state should exhibit noise-induced memory of the spatial propagation profile of the collective oscillations, which is also verified experimentally. Taken together, these results allow us to construe the high regularity observed experimentally in the brain as an instance of collective stochastic coherence.

Reward-based training of recurrent neural networks for cognitive and value-based tasks

PubMed Central

Song, H Francis; Yang, Guangyu R; Wang, Xiao-Jing

2017-01-01

Trained neural network models, which exhibit features of neural activity recorded from behaving animals, may provide insights into the circuit mechanisms of cognitive functions through systematic analysis of network activity and connectivity. However, in contrast to the graded error signals commonly used to train networks through supervised learning, animals learn from reward feedback on definite actions through reinforcement learning. Reward maximization is particularly relevant when optimal behavior depends on an animal’s internal judgment of confidence or subjective preferences. Here, we implement reward-based training of recurrent neural networks in which a value network guides learning by using the activity of the decision network to predict future reward. We show that such models capture behavioral and electrophysiological findings from well-known experimental paradigms. Our work provides a unified framework for investigating diverse cognitive and value-based computations, and predicts a role for value representation that is essential for learning, but not executing, a task. DOI: http://dx.doi.org/10.7554/eLife.21492.001 PMID:28084991

A novel recurrent neural network with finite-time convergence for linear programming.

PubMed

Liu, Qingshan; Cao, Jinde; Chen, Guanrong

2010-11-01

In this letter, a novel recurrent neural network based on the gradient method is proposed for solving linear programming problems. Finite-time convergence of the proposed neural network is proved by using the Lyapunov method. Compared with the existing neural networks for linear programming, the proposed neural network is globally convergent to exact optimal solutions in finite time, which is remarkable and rare in the literature of neural networks for optimization. Some numerical examples are given to show the effectiveness and excellent performance of the new recurrent neural network.

Neural Networks for Rapid Design and Analysis

NASA Technical Reports Server (NTRS)

Sparks, Dean W., Jr.; Maghami, Peiman G.

1998-01-01

Artificial neural networks have been employed for rapid and efficient dynamics and control analysis of flexible systems. Specifically, feedforward neural networks are designed to approximate nonlinear dynamic components over prescribed input ranges, and are used in simulations as a means to speed up the overall time response analysis process. To capture the recursive nature of dynamic components with artificial neural networks, recurrent networks, which use state feedback with the appropriate number of time delays, as inputs to the networks, are employed. Once properly trained, neural networks can give very good approximations to nonlinear dynamic components, and by their judicious use in simulations, allow the analyst the potential to speed up the analysis process considerably. To illustrate this potential speed up, an existing simulation model of a spacecraft reaction wheel system is executed, first conventionally, and then with an artificial neural network in place.

A nonlinear dynamical system for combustion instability in a pulse model combustor

NASA Astrophysics Data System (ADS)

Takagi, Kazushi; Gotoda, Hiroshi

2016-11-01

We theoretically and numerically study the bifurcation phenomena of nonlinear dynamical system describing combustion instability in a pulse model combustor on the basis of dynamical system theory and complex network theory. The dynamical behavior of pressure fluctuations undergoes a significant transition from steady-state to deterministic chaos via the period-doubling cascade process known as Feigenbaum scenario with decreasing the characteristic flow time. Recurrence plots and recurrence networks analysis we adopted in this study can quantify the significant changes in dynamic behavior of combustion instability that cannot be captured in the bifurcation diagram.

Cross-cohort analysis identifies a TEAD4 ↔ MYCN positive-feedback loop as the core regulatory element of high-risk neuroblastoma. | Office of Cancer Genomics

Cancer.gov

High-risk neuroblastomas show a paucity of recurrent somatic mutations at diagnosis. As a result, the molecular basis for this aggressive phenotype remains elusive. Recent progress in regulatory network analysis helped us elucidate disease-driving mechanisms downstream of genomic alterations, including recurrent chromosomal alterations. Our analysis identified three molecular subtypes of high-risk neuroblastomas, consistent with chromosomal alterations, and identified subtype-specific master regulator (MR) proteins that were conserved across independent cohorts.

Deep Recurrent Neural Network-Based Autoencoders for Acoustic Novelty Detection

PubMed Central

Vesperini, Fabio; Schuller, Björn

2017-01-01

In the emerging field of acoustic novelty detection, most research efforts are devoted to probabilistic approaches such as mixture models or state-space models. Only recent studies introduced (pseudo-)generative models for acoustic novelty detection with recurrent neural networks in the form of an autoencoder. In these approaches, auditory spectral features of the next short term frame are predicted from the previous frames by means of Long-Short Term Memory recurrent denoising autoencoders. The reconstruction error between the input and the output of the autoencoder is used as activation signal to detect novel events. There is no evidence of studies focused on comparing previous efforts to automatically recognize novel events from audio signals and giving a broad and in depth evaluation of recurrent neural network-based autoencoders. The present contribution aims to consistently evaluate our recent novel approaches to fill this white spot in the literature and provide insight by extensive evaluations carried out on three databases: A3Novelty, PASCAL CHiME, and PROMETHEUS. Besides providing an extensive analysis of novel and state-of-the-art methods, the article shows how RNN-based autoencoders outperform statistical approaches up to an absolute improvement of 16.4% average F-measure over the three databases. PMID:28182121

Emergence of network structure due to spike-timing-dependent plasticity in recurrent neuronal networks IV: structuring synaptic pathways among recurrent connections.

PubMed

Gilson, Matthieu; Burkitt, Anthony N; Grayden, David B; Thomas, Doreen A; van Hemmen, J Leo

2009-12-01

In neuronal networks, the changes of synaptic strength (or weight) performed by spike-timing-dependent plasticity (STDP) are hypothesized to give rise to functional network structure. This article investigates how this phenomenon occurs for the excitatory recurrent connections of a network with fixed input weights that is stimulated by external spike trains. We develop a theoretical framework based on the Poisson neuron model to analyze the interplay between the neuronal activity (firing rates and the spike-time correlations) and the learning dynamics, when the network is stimulated by correlated pools of homogeneous Poisson spike trains. STDP can lead to both a stabilization of all the neuron firing rates (homeostatic equilibrium) and a robust weight specialization. The pattern of specialization for the recurrent weights is determined by a relationship between the input firing-rate and correlation structures, the network topology, the STDP parameters and the synaptic response properties. We find conditions for feed-forward pathways or areas with strengthened self-feedback to emerge in an initially homogeneous recurrent network.

Learning State Space Dynamics in Recurrent Networks

NASA Astrophysics Data System (ADS)

Simard, Patrice Yvon

Fully recurrent (asymmetrical) networks can be used to learn temporal trajectories. The network is unfolded in time, and backpropagation is used to train the weights. The presence of recurrent connections creates internal states in the system which vary as a function of time. The resulting dynamics can provide interesting additional computing power but learning is made more difficult by the existence of internal memories. This study first exhibits the properties of recurrent networks in terms of convergence when the internal states of the system are unknown. A new energy functional is provided to change the weights of the units in order to the control the stability of the fixed points of the network's dynamics. The power of the resultant algorithm is illustrated with the simulation of a content addressable memory. Next, the more general case of time trajectories on a recurrent network is studied. An application is proposed in which trajectories are generated to draw letters as a function of an input. In another application of recurrent systems, a neural network certain temporal properties observed in human callosally sectioned brains. Finally the proposed algorithm for stabilizing dynamics around fixed points is extended to one for stabilizing dynamics around time trajectories. Its effects are illustrated on a network which generates Lisajous curves.

Artificial intelligence for predicting recurrence-free probability of non-invasive high-grade urothelial bladder cell carcinoma.

PubMed

Cai, Tommaso; Conti, Gloria; Nesi, Gabriella; Lorenzini, Matteo; Mondaini, Nicola; Bartoletti, Riccardo

2007-10-01

The objective of our study was to define a neural network for predicting recurrence and progression-free probability in patients affected by recurrent pTaG3 urothelial bladder cancer to use in everyday clinical practice. Among all patients who had undergone transurethral resection for bladder tumors, 143 were finally selected and enrolled. Four follow-ups for recurrence, progression or survival were performed at 6, 9, 12 and 108 months. The data were analyzed by using the commercially available software program NeuralWorks Predict. These data were compared with univariate and multivariate analysis results. The use of Artificial Neural Networks (ANN) in recurrent pTaG3 patients showed a sensitivity of 81.67% and specificity of 95.87% in predicting recurrence-free status after transurethral resection of bladder tumor at 12 months follow-up. Statistical and ANN analyses allowed selection of the number of lesions (multiple, HR=3.31, p=0.008) and the previous recurrence rate (>or=2/year, HR=3.14, p=0.003) as the most influential variables affecting the output decision in predicting the natural history of recurrent pTaG3 urothelial bladder cancer. ANN applications also included selection of the previous adjuvant therapy. We demonstrated the feasibility and reliability of ANN applications in everyday clinical practice, reporting a good recurrence predicting performance. The study identified a single subgroup of pTaG3 patients with multiple lesions, >or=2/year recurrence rate and without any response to previous Bacille Calmette-Guérin adjuvant therapy, that seem to be at high risk of recurrence.

Synthesis of recurrent neural networks for dynamical system simulation.

PubMed

Trischler, Adam P; D'Eleuterio, Gabriele M T

2016-08-01

We review several of the most widely used techniques for training recurrent neural networks to approximate dynamical systems, then describe a novel algorithm for this task. The algorithm is based on an earlier theoretical result that guarantees the quality of the network approximation. We show that a feedforward neural network can be trained on the vector-field representation of a given dynamical system using backpropagation, then recast it as a recurrent network that replicates the original system's dynamics. After detailing this algorithm and its relation to earlier approaches, we present numerical examples that demonstrate its capabilities. One of the distinguishing features of our approach is that both the original dynamical systems and the recurrent networks that simulate them operate in continuous time. Copyright © 2016 Elsevier Ltd. All rights reserved.

Antiplatelet Agents for the Secondary Prevention of Ischemic Stroke or Transient Ischemic Attack: A Network Meta-Analysis.

PubMed

Wang, Wen; Zhang, Lu; Liu, Weiming; Zhu, Qin; Lan, Qing; Zhao, Jizong

2016-05-01

Stroke can cause high morbidity and mortality, and ischemic stroke (IS) and transient ischemic attack (TIA) patients have a high stroke recurrence rate. Antiplatelet agents are the standard therapy for these patients, but it is often difficult for clinicians to select the best therapy from among the multiple treatment options. We therefore performed a network meta-analysis to estimate the efficacy of antiplatelet agents for secondary prevention of recurrent stroke. We systematically searched 3 databases (PubMed, Embase, and Cochrane) for relevant studies published through August 2015. The primary end points of this meta-analysis were overall stroke, hemorrhagic stroke, and fatal stroke. A total of 30 trials were included in our network meta-analysis and abstracted data. Among the therapies evaluated in the included trials, the estimates for overall stroke and hemorrhagic stroke for cilostazol (Cilo) were significantly better than those for aspirin (odds ratio [OR] = .64, 95% credibility interval [CrI], .45-.91; OR = .23, 95% CrI, .08-.58). The estimate for fatal stroke was highest for Cilo plus aspirin combination therapy, followed by Cilo therapy. The results of our meta-analysis indicate that Cilo significantly improves overall stroke and hemorrhagic stroke in IS or TIA patients and reduces fatal stroke, but with low statistical significance. Our results also show that Cilo was significantly more efficient than other therapies in Asian patients; therefore, future trials should focus on Cilo treatment for secondary prevention of recurrent stroke in non-Asian patients. Copyright © 2016 National Stroke Association. Published by Elsevier Inc. All rights reserved.

A novel approach to the dynamical complexity of the Earth's magnetosphere at geomagnetic storm time-scales based on recurrences

NASA Astrophysics Data System (ADS)

Donner, Reik; Balasis, Georgios; Stolbova, Veronika; Wiedermann, Marc; Georgiou, Marina; Kurths, Jürgen

2016-04-01

Magnetic storms are the most prominent global manifestations of out-of-equilibrium magnetospheric dynamics. Investigating the dynamical complexity exhibited by geomagnetic observables can provide valuable insights into relevant physical processes as well as temporal scales associated with this phenomenon. In this work, we introduce several innovative data analysis techniques enabling a quantitative analysis of the Dst index non-stationary behavior. Using recurrence quantification analysis (RQA) and recurrence network analysis (RNA), we obtain a variety of complexity measures serving as markers of quiet- and storm-time magnetospheric dynamics. We additionally apply these techniques to the main driver of Dst index variations, the V BSouth coupling function and interplanetary medium parameters Bz and Pdyn in order to discriminate internal processes from the magnetosphere's response directly induced by the external forcing by the solar wind. The derived recurrence-based measures allow us to improve the accuracy with which magnetospheric storms can be classified based on ground-based observations. The new methodology presented here could be of significant interest for the space weather research community working on time series analysis for magnetic storm forecasts.

Homeostatic Scaling of Excitability in Recurrent Neural Networks

PubMed Central

Remme, Michiel W. H.; Wadman, Wytse J.

2012-01-01

Neurons adjust their intrinsic excitability when experiencing a persistent change in synaptic drive. This process can prevent neural activity from moving into either a quiescent state or a saturated state in the face of ongoing plasticity, and is thought to promote stability of the network in which neurons reside. However, most neurons are embedded in recurrent networks, which require a delicate balance between excitation and inhibition to maintain network stability. This balance could be disrupted when neurons independently adjust their intrinsic excitability. Here, we study the functioning of activity-dependent homeostatic scaling of intrinsic excitability (HSE) in a recurrent neural network. Using both simulations of a recurrent network consisting of excitatory and inhibitory neurons that implement HSE, and a mean-field description of adapting excitatory and inhibitory populations, we show that the stability of such adapting networks critically depends on the relationship between the adaptation time scales of both neuron populations. In a stable adapting network, HSE can keep all neurons functioning within their dynamic range, while the network is undergoing several (patho)physiologically relevant types of plasticity, such as persistent changes in external drive, changes in connection strengths, or the loss of inhibitory cells from the network. However, HSE cannot prevent the unstable network dynamics that result when, due to such plasticity, recurrent excitation in the network becomes too strong compared to feedback inhibition. This suggests that keeping a neural network in a stable and functional state requires the coordination of distinct homeostatic mechanisms that operate not only by adjusting neural excitability, but also by controlling network connectivity. PMID:22570604

Albendazole and Corticosteroids for the Treatment of Solitary Cysticercus Granuloma: A Network Meta-analysis.

PubMed

Zhao, Bing-Cheng; Jiang, Hong-Ye; Ma, Wei-Ying; Jin, Da-Di; Li, Hao-Miao; Lu, Hai; Nakajima, Hideaki; Huang, Tong-Yi; Sun, Kai-Yu; Chen, Shu-Ling; Chen, Ke-Bing

2016-02-01

Solitary cysticercus granuloma (SCG) is the commonest form of neurocysticercosis in the Indian subcontinent and in travelers. Several different treatment options exist for SCG. We conducted a Bayesian network meta-analysis of randomized clinical trials (RCTs) to identify the best treatment option to prevent seizure recurrence and promote lesion resolution for patients with SCG. PubMed, EMBASE and the Cochrane Library databases (up to June 1, 2015) were searched for RCTs that compared any anthelmintics or corticosteroids, alone or in combination, with placebo or head to head and reported on seizure recurrence and lesion resolution in patients with SCG. A total of 14 RCTs (1277 patients) were included in the quantitative analysis focusing on four different treatment options. A Bayesian network model computing odds ratios (OR) with 95% credible intervals (CrI) and probability of being best (Pbest) was used to compare all interventions simultaneously. Albendazole and corticosteroids combination therapy was the only regimen that significantly decreased the risk of seizure recurrence compared with conservative treatment (OR 0.32, 95% CrI 0.10-0.93, Pbest 73.3%). Albendazole and corticosteroids alone or in combination were all efficacious in hastening granuloma resolution, but the combined therapy remained the best option based on probability analysis (OR 3.05, 95% CrI 1.24-7.95, Pbest 53.9%). The superiority of the combination therapy changed little in RCTs with different follow-up durations and in sensitivity analyses. The limitations of this study include high risk of bias and short follow-up duration in most studies. Dual therapy of albendazole and corticosteroids was the most efficacious regimen that could prevent seizure recurrence and promote lesion resolution in a follow-up period of around one year. It should be recommended for the management of SCG until more high-quality evidence is available.

Albendazole and Corticosteroids for the Treatment of Solitary Cysticercus Granuloma: A Network Meta-analysis

PubMed Central

Nakajima, Hideaki; Huang, Tong-Yi; Sun, Kai-Yu; Chen, Shu-Ling; Chen, Ke-Bing

2016-01-01

Background Solitary cysticercus granuloma (SCG) is the commonest form of neurocysticercosis in the Indian subcontinent and in travelers. Several different treatment options exist for SCG. We conducted a Bayesian network meta-analysis of randomized clinical trials (RCTs) to identify the best treatment option to prevent seizure recurrence and promote lesion resolution for patients with SCG. Methods and Principal Findings PubMed, EMBASE and the Cochrane Library databases (up to June 1, 2015) were searched for RCTs that compared any anthelmintics or corticosteroids, alone or in combination, with placebo or head to head and reported on seizure recurrence and lesion resolution in patients with SCG. A total of 14 RCTs (1277 patients) were included in the quantitative analysis focusing on four different treatment options. A Bayesian network model computing odds ratios (OR) with 95% credible intervals (CrI) and probability of being best (Pbest) was used to compare all interventions simultaneously. Albendazole and corticosteroids combination therapy was the only regimen that significantly decreased the risk of seizure recurrence compared with conservative treatment (OR 0.32, 95% CrI 0.10–0.93, Pbest 73.3%). Albendazole and corticosteroids alone or in combination were all efficacious in hastening granuloma resolution, but the combined therapy remained the best option based on probability analysis (OR 3.05, 95% CrI 1.24–7.95, Pbest 53.9%). The superiority of the combination therapy changed little in RCTs with different follow-up durations and in sensitivity analyses. The limitations of this study include high risk of bias and short follow-up duration in most studies. Conclusions Dual therapy of albendazole and corticosteroids was the most efficacious regimen that could prevent seizure recurrence and promote lesion resolution in a follow-up period of around one year. It should be recommended for the management of SCG until more high-quality evidence is available. PMID:26849048

Robust passivity analysis for discrete-time recurrent neural networks with mixed delays

NASA Astrophysics Data System (ADS)

Huang, Chuan-Kuei; Shu, Yu-Jeng; Chang, Koan-Yuh; Shou, Ho-Nien; Lu, Chien-Yu

2015-02-01

This article considers the robust passivity analysis for a class of discrete-time recurrent neural networks (DRNNs) with mixed time-delays and uncertain parameters. The mixed time-delays that consist of both the discrete time-varying and distributed time-delays in a given range are presented, and the uncertain parameters are norm-bounded. The activation functions are assumed to be globally Lipschitz continuous. Based on new bounding technique and appropriate type of Lyapunov functional, a sufficient condition is investigated to guarantee the existence of the desired robust passivity condition for the DRNNs, which can be derived in terms of a family of linear matrix inequality (LMI). Some free-weighting matrices are introduced to reduce the conservatism of the criterion by using the bounding technique. A numerical example is given to illustrate the effectiveness and applicability.

Recurrent hepatocellular carcinoma after liver transplant: identifying the high-risk patient

PubMed Central

Nissen, Nicholas N; Menon, Vijay; Bresee, Catherine; Tran, Tram T; Annamalai, Alagappan; Poordad, Fred; Fair, Jeffrey H; Klein, Andrew S; Boland, Brendan; Colquhoun, Steven D

2011-01-01

Background Recurrence of hepatocellular carcinoma (HCC) after liver transplantation (LT) is rarely curable. However, in view of the advent of new treatments, it is critical that patients at high risk for recurrence are identified. Methods Patients undergoing LT for HCC at a single centre between 2002 and 2010 were reviewed and data on clinical parameters and explant pathology were analysed to determine factors associated with HCC recurrence. All necrotic and viable tumour nodules were included in explant staging. All patients underwent LT according to the United Network for Organ Sharing (UNOS) Model for End-stage Liver Disease (MELD) tumour exception policies. Results Liver transplantation was performed in 122 patients with HCC during this period. Rates of recurrence-free survival in the entire cohort at 1 year and 3 years were 95% and 89%, respectively. Thirteen patients developed HCC recurrence at a median of 14 months post-LT. In univariate analysis the factors associated with HCC recurrence were bilobar tumours, vascular invasion, and stage exceeding either Milan or University of California San Francisco (UCSF) Criteria. Multivariate analysis showed pathology outside UCSF Criteria was the major predictor of recurrence; when pathology outside UCSF Criteria was found in combination with vascular invasion, the predicted 3-year recurrence-free survival was only 26%. Conclusions Explant pathology can be used to predict the risk for recurrent HCC after LT, which may allow for improved adjuvant and management strategies. PMID:21843263

Biologically plausible learning in recurrent neural networks reproduces neural dynamics observed during cognitive tasks

PubMed Central

Miconi, Thomas

2017-01-01

Neural activity during cognitive tasks exhibits complex dynamics that flexibly encode task-relevant variables. Chaotic recurrent networks, which spontaneously generate rich dynamics, have been proposed as a model of cortical computation during cognitive tasks. However, existing methods for training these networks are either biologically implausible, and/or require a continuous, real-time error signal to guide learning. Here we show that a biologically plausible learning rule can train such recurrent networks, guided solely by delayed, phasic rewards at the end of each trial. Networks endowed with this learning rule can successfully learn nontrivial tasks requiring flexible (context-dependent) associations, memory maintenance, nonlinear mixed selectivities, and coordination among multiple outputs. The resulting networks replicate complex dynamics previously observed in animal cortex, such as dynamic encoding of task features and selective integration of sensory inputs. We conclude that recurrent neural networks offer a plausible model of cortical dynamics during both learning and performance of flexible behavior. DOI: http://dx.doi.org/10.7554/eLife.20899.001 PMID:28230528

Biologically plausible learning in recurrent neural networks reproduces neural dynamics observed during cognitive tasks.

PubMed

Miconi, Thomas

2017-02-23

Neural activity during cognitive tasks exhibits complex dynamics that flexibly encode task-relevant variables. Chaotic recurrent networks, which spontaneously generate rich dynamics, have been proposed as a model of cortical computation during cognitive tasks. However, existing methods for training these networks are either biologically implausible, and/or require a continuous, real-time error signal to guide learning. Here we show that a biologically plausible learning rule can train such recurrent networks, guided solely by delayed, phasic rewards at the end of each trial. Networks endowed with this learning rule can successfully learn nontrivial tasks requiring flexible (context-dependent) associations, memory maintenance, nonlinear mixed selectivities, and coordination among multiple outputs. The resulting networks replicate complex dynamics previously observed in animal cortex, such as dynamic encoding of task features and selective integration of sensory inputs. We conclude that recurrent neural networks offer a plausible model of cortical dynamics during both learning and performance of flexible behavior.

Spike timing analysis in neural networks with unsupervised synaptic plasticity

NASA Astrophysics Data System (ADS)

Mizusaki, B. E. P.; Agnes, E. J.; Brunnet, L. G.; Erichsen, R., Jr.

2013-01-01

The synaptic plasticity rules that sculpt a neural network architecture are key elements to understand cortical processing, as they may explain the emergence of stable, functional activity, while avoiding runaway excitation. For an associative memory framework, they should be built in a way as to enable the network to reproduce a robust spatio-temporal trajectory in response to an external stimulus. Still, how these rules may be implemented in recurrent networks and the way they relate to their capacity of pattern recognition remains unclear. We studied the effects of three phenomenological unsupervised rules in sparsely connected recurrent networks for associative memory: spike-timing-dependent-plasticity, short-term-plasticity and an homeostatic scaling. The system stability is monitored during the learning process of the network, as the mean firing rate converges to a value determined by the homeostatic scaling. Afterwards, it is possible to measure the recovery efficiency of the activity following each initial stimulus. This is evaluated by a measure of the correlation between spike fire timings, and we analysed the full memory separation capacity and limitations of this system.

A novel nonlinear adaptive filter using a pipelined second-order Volterra recurrent neural network.

PubMed

Zhao, Haiquan; Zhang, Jiashu

2009-12-01

To enhance the performance and overcome the heavy computational complexity of recurrent neural networks (RNN), a novel nonlinear adaptive filter based on a pipelined second-order Volterra recurrent neural network (PSOVRNN) is proposed in this paper. A modified real-time recurrent learning (RTRL) algorithm of the proposed filter is derived in much more detail. The PSOVRNN comprises of a number of simple small-scale second-order Volterra recurrent neural network (SOVRNN) modules. In contrast to the standard RNN, these modules of a PSOVRNN can be performed simultaneously in a pipelined parallelism fashion, which can lead to a significant improvement in its total computational efficiency. Moreover, since each module of the PSOVRNN is a SOVRNN in which nonlinearity is introduced by the recursive second-order Volterra (RSOV) expansion, its performance can be further improved. Computer simulations have demonstrated that the PSOVRNN performs better than the pipelined recurrent neural network (PRNN) and RNN for nonlinear colored signals prediction and nonlinear channel equalization. However, the superiority of the PSOVRNN over the PRNN is at the cost of increasing computational complexity due to the introduced nonlinear expansion of each module.

An evolutionary algorithm that constructs recurrent neural networks.

PubMed

Angeline, P J; Saunders, G M; Pollack, J B

1994-01-01

Standard methods for simultaneously inducing the structure and weights of recurrent neural networks limit every task to an assumed class of architectures. Such a simplification is necessary since the interactions between network structure and function are not well understood. Evolutionary computations, which include genetic algorithms and evolutionary programming, are population-based search methods that have shown promise in many similarly complex tasks. This paper argues that genetic algorithms are inappropriate for network acquisition and describes an evolutionary program, called GNARL, that simultaneously acquires both the structure and weights for recurrent networks. GNARL's empirical acquisition method allows for the emergence of complex behaviors and topologies that are potentially excluded by the artificial architectural constraints imposed in standard network induction methods.

Ultrasonographic Diagnosis of Cirrhosis Based on Preprocessing Using Pyramid Recurrent Neural Network

NASA Astrophysics Data System (ADS)

Lu, Jianming; Liu, Jiang; Zhao, Xueqin; Yahagi, Takashi

In this paper, a pyramid recurrent neural network is applied to characterize the hepatic parenchymal diseases in ultrasonic B-scan texture. The cirrhotic parenchymal diseases are classified into 4 types according to the size of hypoechoic nodular lesions. The B-mode patterns are wavelet transformed , and then the compressed data are feed into a pyramid neural network to diagnose the type of cirrhotic diseases. Compared with the 3-layer neural networks, the performance of the proposed pyramid recurrent neural network is improved by utilizing the lower layer effectively. The simulation result shows that the proposed system is suitable for diagnosis of cirrhosis diseases.

How Travel Demand Affects Detection of Non-Recurrent Traffic Congestion on Urban Road Networks

NASA Astrophysics Data System (ADS)

Anbaroglu, B.; Heydecker, B.; Cheng, T.

2016-06-01

Occurrence of non-recurrent traffic congestion hinders the economic activity of a city, as travellers could miss appointments or be late for work or important meetings. Similarly, for shippers, unexpected delays may disrupt just-in-time delivery and manufacturing processes, which could lose them payment. Consequently, research on non-recurrent congestion detection on urban road networks has recently gained attention. By analysing large amounts of traffic data collected on a daily basis, traffic operation centres can improve their methods to detect non-recurrent congestion rapidly and then revise their existing plans to mitigate its effects. Space-time clusters of high link journey time estimates correspond to non-recurrent congestion events. Existing research, however, has not considered the effect of travel demand on the effectiveness of non-recurrent congestion detection methods. Therefore, this paper investigates how travel demand affects detection of non-recurrent traffic congestion detection on urban road networks. Travel demand has been classified into three categories as low, normal and high. The experiments are carried out on London's urban road network, and the results demonstrate the necessity to adjust the relative importance of the component evaluation criteria depending on the travel demand level.

Predictors of incident and recurrent participation in the sale or delivery of drugs for profit amongst young methamphetamine users in Chiang Mai Province, Thailand, 2005-2006.

PubMed

Latimore, Amanda D; Rudolph, Abby; German, Danielle; Sherman, Susan G; Srirojn, Bangorn; Aramrattana, Apinun; Celentano, David D

2011-07-01

Despite Thailand's war on drugs, methamphetamine ("yaba" in Thai) use and the drug economy both thrive. This analysis identifies predictors of incident and recurrent involvement in the sale or delivery of drugs for profit amongst young Thai yaba users. Between April 2005 and June 2006, 983 yaba users, ages 18-25, were enrolled in a randomized behavioural intervention in Chiang Mai Province (415 index and 568 of their drug network members). Questionnaires administered at baseline, 3-, 6-, 9-, and 12-month follow-up visits assessed socio-demographic factors, current and prior drug use, social network characteristics, sexual risk behaviours and drug use norms. Exposures were lagged by three months (prior visit). Outcomes included incident and recurrent drug economy involvement. Generalized linear mixed models were fit using GLIMMIX (SASv9.1). Incident drug economy involvement was predicted by yaba use frequency (adjusted odds ratio [AOR]: 1.05; 95% confidence interval [CI]: 1.01, 1.10), recent incarceration (AOR: 2.37; 95% CI: 1.07, 5.25) and the proportion of yaba-using networks who quit recently (AOR: .34; 95% CI: .15, .78). Recurrent drug economy involvement was predicted by age (AOR: 0.81; 95% CI: 0.68, 0.96), frequency of yaba use (AOR: 1.06; 95% CI: 1.02, 1.09), drug economy involvement at the previous visit (AOR: 2.61; CI: 1.59, 4.28), incarceration in the prior three months (AOR: 2.29; 95% CI: 1.07, 4.86), and the proportion of yaba-users in his/her network who quit recently (AOR: .38; 95% CI: .20, .71). Individual drug use, drug use in social networks and recent incarceration were predictors of incident and recurrent involvement in the drug economy. These results suggest that interrupting drug use and/or minimizing the influence of drug-using networks may help prevent further involvement in the drug economy. The emergence of recent incarceration as a predictor for both models highlights the need for more appropriate drug rehabilitation programmes and demonstrates that continued criminalization of drug users may fuel Thailand's yaba epidemic. Copyright © 2011 Elsevier B.V. All rights reserved.

Protein secondary structure prediction using modular reciprocal bidirectional recurrent neural networks.

PubMed

Babaei, Sepideh; Geranmayeh, Amir; Seyyedsalehi, Seyyed Ali

2010-12-01

The supervised learning of recurrent neural networks well-suited for prediction of protein secondary structures from the underlying amino acids sequence is studied. Modular reciprocal recurrent neural networks (MRR-NN) are proposed to model the strong correlations between adjacent secondary structure elements. Besides, a multilayer bidirectional recurrent neural network (MBR-NN) is introduced to capture the long-range intramolecular interactions between amino acids in formation of the secondary structure. The final modular prediction system is devised based on the interactive integration of the MRR-NN and the MBR-NN structures to arbitrarily engage the neighboring effects of the secondary structure types concurrent with memorizing the sequential dependencies of amino acids along the protein chain. The advanced combined network augments the percentage accuracy (Q₃) to 79.36% and boosts the segment overlap (SOV) up to 70.09% when tested on the PSIPRED dataset in three-fold cross-validation. Copyright © 2010 Elsevier Ireland Ltd. All rights reserved.

Ridge Polynomial Neural Network with Error Feedback for Time Series Forecasting

PubMed Central

Ghazali, Rozaida; Herawan, Tutut

2016-01-01

Time series forecasting has gained much attention due to its many practical applications. Higher-order neural network with recurrent feedback is a powerful technique that has been used successfully for time series forecasting. It maintains fast learning and the ability to learn the dynamics of the time series over time. Network output feedback is the most common recurrent feedback for many recurrent neural network models. However, not much attention has been paid to the use of network error feedback instead of network output feedback. In this study, we propose a novel model, called Ridge Polynomial Neural Network with Error Feedback (RPNN-EF) that incorporates higher order terms, recurrence and error feedback. To evaluate the performance of RPNN-EF, we used four univariate time series with different forecasting horizons, namely star brightness, monthly smoothed sunspot numbers, daily Euro/Dollar exchange rate, and Mackey-Glass time-delay differential equation. We compared the forecasting performance of RPNN-EF with the ordinary Ridge Polynomial Neural Network (RPNN) and the Dynamic Ridge Polynomial Neural Network (DRPNN). Simulation results showed an average 23.34% improvement in Root Mean Square Error (RMSE) with respect to RPNN and an average 10.74% improvement with respect to DRPNN. That means that using network errors during training helps enhance the overall forecasting performance for the network. PMID:27959927

Ridge Polynomial Neural Network with Error Feedback for Time Series Forecasting.

PubMed

Waheeb, Waddah; Ghazali, Rozaida; Herawan, Tutut

2016-01-01

Time series forecasting has gained much attention due to its many practical applications. Higher-order neural network with recurrent feedback is a powerful technique that has been used successfully for time series forecasting. It maintains fast learning and the ability to learn the dynamics of the time series over time. Network output feedback is the most common recurrent feedback for many recurrent neural network models. However, not much attention has been paid to the use of network error feedback instead of network output feedback. In this study, we propose a novel model, called Ridge Polynomial Neural Network with Error Feedback (RPNN-EF) that incorporates higher order terms, recurrence and error feedback. To evaluate the performance of RPNN-EF, we used four univariate time series with different forecasting horizons, namely star brightness, monthly smoothed sunspot numbers, daily Euro/Dollar exchange rate, and Mackey-Glass time-delay differential equation. We compared the forecasting performance of RPNN-EF with the ordinary Ridge Polynomial Neural Network (RPNN) and the Dynamic Ridge Polynomial Neural Network (DRPNN). Simulation results showed an average 23.34% improvement in Root Mean Square Error (RMSE) with respect to RPNN and an average 10.74% improvement with respect to DRPNN. That means that using network errors during training helps enhance the overall forecasting performance for the network.

Gradient calculations for dynamic recurrent neural networks: a survey.

PubMed

Pearlmutter, B A

1995-01-01

Surveys learning algorithms for recurrent neural networks with hidden units and puts the various techniques into a common framework. The authors discuss fixed point learning algorithms, namely recurrent backpropagation and deterministic Boltzmann machines, and nonfixed point algorithms, namely backpropagation through time, Elman's history cutoff, and Jordan's output feedback architecture. Forward propagation, an on-line technique that uses adjoint equations, and variations thereof, are also discussed. In many cases, the unified presentation leads to generalizations of various sorts. The author discusses advantages and disadvantages of temporally continuous neural networks in contrast to clocked ones continues with some "tricks of the trade" for training, using, and simulating continuous time and recurrent neural networks. The author presents some simulations, and at the end, addresses issues of computational complexity and learning speed.

Analyzing long-term correlated stochastic processes by means of recurrence networks: Potentials and pitfalls

NASA Astrophysics Data System (ADS)

Zou, Yong; Donner, Reik V.; Kurths, Jürgen

2015-02-01

Long-range correlated processes are ubiquitous, ranging from climate variables to financial time series. One paradigmatic example for such processes is fractional Brownian motion (fBm). In this work, we highlight the potentials and conceptual as well as practical limitations when applying the recently proposed recurrence network (RN) approach to fBm and related stochastic processes. In particular, we demonstrate that the results of a previous application of RN analysis to fBm [Liu et al. Phys. Rev. E 89, 032814 (2014), 10.1103/PhysRevE.89.032814] are mainly due to an inappropriate treatment disregarding the intrinsic nonstationarity of such processes. Complementarily, we analyze some RN properties of the closely related stationary fractional Gaussian noise (fGn) processes and find that the resulting network properties are well-defined and behave as one would expect from basic conceptual considerations. Our results demonstrate that RN analysis can indeed provide meaningful results for stationary stochastic processes, given a proper selection of its intrinsic methodological parameters, whereas it is prone to fail to uniquely retrieve RN properties for nonstationary stochastic processes like fBm.

A statistical framework for evaluating neural networks to predict recurrent events in breast cancer

NASA Astrophysics Data System (ADS)

Gorunescu, Florin; Gorunescu, Marina; El-Darzi, Elia; Gorunescu, Smaranda

2010-07-01

Breast cancer is the second leading cause of cancer deaths in women today. Sometimes, breast cancer can return after primary treatment. A medical diagnosis of recurrent cancer is often a more challenging task than the initial one. In this paper, we investigate the potential contribution of neural networks (NNs) to support health professionals in diagnosing such events. The NN algorithms are tested and applied to two different datasets. An extensive statistical analysis has been performed to verify our experiments. The results show that a simple network structure for both the multi-layer perceptron and radial basis function can produce equally good results, not all attributes are needed to train these algorithms and, finally, the classification performances of all algorithms are statistically robust. Moreover, we have shown that the best performing algorithm will strongly depend on the features of the datasets, and hence, there is not necessarily a single best classifier.

Proteomics and bioinformatics analysis of altered protein expression in the placental villous tissue from early recurrent miscarriage patients.

PubMed

Pan, Hai-Tao; Ding, Hai-Gang; Fang, Min; Yu, Bin; Cheng, Yi; Tan, Ya-Jing; Fu, Qi-Qin; Lu, Bo; Cai, Hong-Guang; Jin, Xin; Xia, Xian-Qing; Zhang, Tao

2018-01-01

Recurrent miscarriage (RM) affects 5% of women, it has an adverse emotional impact on women. Because of the complexities of early development, the mechanism of recurrent miscarriage is still unclear. We hypothesized that abnormal placenta leads to early recurrent miscarriage (ERM). The aim of this study was to identify ERM associated factors in human placenta villous tissue using proteomics. Investigation of these differences in protein expression in parallel profiling is essential to understand the comprehensive pathophysiological mechanism underlying recurrent miscarriage (RM). To gain more insight into mechanisms of recurrent miscarriage (RM), a comparative proteome profile of the human placenta villous tissue in normal and RM pregnancies was analyzed using iTRAQ technology and bioinformatics analysis used by Ingenuity Pathway Analysis (IPA) software. In this study, we employed an iTRAQ based proteomics analysis of four placental villous tissues from patients with early recurrent miscarriage (ERM) and four from normal pregnant women. Finally, we identified 2805 proteins and 79,998 peptides between patients with RM and normal matched group. Further analysis identified 314 differentially expressed proteins in placental villous tissue (≥1.3-fold, Student's t-test, p < 0.05); 209 proteins showed the increased expression while 105 proteins showed decreased expression. These 314 proteins were analyzed by Ingenuity Pathway Analysis (IPA) and were found to play important roles in the growth of embryo. Furthermore, network analysis show that Angiotensinogen (AGT), MAPK14 and Prothrombin (F2) are core factors in early embryonic development. We used another 8 independent samples (4 cases and 4 controls) to cross validation of the proteomic data. This study has identified several proteins that are associated with early development, these results may supply new insight into mechanisms behind recurrent miscarriage. Copyright © 2017 Elsevier Ltd. All rights reserved.

Genesis of interictal spikes in the CA1: a computational investigation

PubMed Central

Ratnadurai-Giridharan, Shivakeshavan; Stefanescu, Roxana A.; Khargonekar, Pramod P.; Carney, Paul R.; Talathi, Sachin S.

2014-01-01

Interictal spikes (IISs) are spontaneous high amplitude, short time duration <400 ms events often observed in electroencephalographs (EEG) of epileptic patients. In vitro analysis of resected mesial temporal lobe tissue from patients with refractory temporal lobe epilepsy has revealed the presence of IIS in the CA1 subfield. In this paper, we develop a biophysically relevant network model of the CA1 subfield and investigate how changes in the network properties influence the susceptibility of CA1 to exhibit an IIS. We present a novel template based approach to identify conditions under which synchronization of paroxysmal depolarization shift (PDS) events evoked in CA1 pyramidal (Py) cells can trigger an IIS. The results from this analysis are used to identify the synaptic parameters of a minimal network model that is capable of generating PDS in response to afferent synaptic input. The minimal network model parameters are then incorporated into a detailed network model of the CA1 subfield in order to address the following questions: (1) How does the formation of an IIS in the CA1 depend on the degree of sprouting (recurrent connections) between the CA1 Py cells and the fraction of CA3 Shaffer collateral (SC) connections onto the CA1 Py cells? and (2) Is synchronous afferent input from the SC essential for the CA1 to exhibit IIS? Our results suggest that the CA1 subfield with low recurrent connectivity (absence of sprouting), mimicking the topology of a normal brain, has a very low probability of producing an IIS except when a large fraction of CA1 neurons (>80%) receives a barrage of quasi-synchronous afferent input (input occurring within a temporal window of ≤24 ms) via the SC. However, as we increase the recurrent connectivity of the CA1 (Psprout > 40); mimicking sprouting in a pathological CA1 network, the CA1 can exhibit IIS even in the absence of a barrage of quasi-synchronous afferents from the SC (input occurring within temporal window >80 ms) and a low fraction of CA1 Py cells (≈30%) receiving SC input. Furthermore, we find that in the presence of Poisson distributed random input via SC, the CA1 network is able to generate spontaneous periodic IISs (≈3 Hz) for high degrees of recurrent Py connectivity (Psprout > 70). We investigate the conditions necessary for this phenomenon and find that spontaneous IISs closely depend on the degree of the network's intrinsic excitability. PMID:24478636

Genesis of interictal spikes in the CA1: a computational investigation.

PubMed

Ratnadurai-Giridharan, Shivakeshavan; Stefanescu, Roxana A; Khargonekar, Pramod P; Carney, Paul R; Talathi, Sachin S

2014-01-01

Interictal spikes (IISs) are spontaneous high amplitude, short time duration <400 ms events often observed in electroencephalographs (EEG) of epileptic patients. In vitro analysis of resected mesial temporal lobe tissue from patients with refractory temporal lobe epilepsy has revealed the presence of IIS in the CA1 subfield. In this paper, we develop a biophysically relevant network model of the CA1 subfield and investigate how changes in the network properties influence the susceptibility of CA1 to exhibit an IIS. We present a novel template based approach to identify conditions under which synchronization of paroxysmal depolarization shift (PDS) events evoked in CA1 pyramidal (Py) cells can trigger an IIS. The results from this analysis are used to identify the synaptic parameters of a minimal network model that is capable of generating PDS in response to afferent synaptic input. The minimal network model parameters are then incorporated into a detailed network model of the CA1 subfield in order to address the following questions: (1) How does the formation of an IIS in the CA1 depend on the degree of sprouting (recurrent connections) between the CA1 Py cells and the fraction of CA3 Shaffer collateral (SC) connections onto the CA1 Py cells? and (2) Is synchronous afferent input from the SC essential for the CA1 to exhibit IIS? Our results suggest that the CA1 subfield with low recurrent connectivity (absence of sprouting), mimicking the topology of a normal brain, has a very low probability of producing an IIS except when a large fraction of CA1 neurons (>80%) receives a barrage of quasi-synchronous afferent input (input occurring within a temporal window of ≤24 ms) via the SC. However, as we increase the recurrent connectivity of the CA1 (P sprout > 40); mimicking sprouting in a pathological CA1 network, the CA1 can exhibit IIS even in the absence of a barrage of quasi-synchronous afferents from the SC (input occurring within temporal window >80 ms) and a low fraction of CA1 Py cells (≈30%) receiving SC input. Furthermore, we find that in the presence of Poisson distributed random input via SC, the CA1 network is able to generate spontaneous periodic IISs (≈3 Hz) for high degrees of recurrent Py connectivity (P sprout > 70). We investigate the conditions necessary for this phenomenon and find that spontaneous IISs closely depend on the degree of the network's intrinsic excitability.

Ads' click-through rates predicting based on gated recurrent unit neural networks

NASA Astrophysics Data System (ADS)

Chen, Qiaohong; Guo, Zixuan; Dong, Wen; Jin, Lingzi

2018-05-01

In order to improve the effect of online advertising and to increase the revenue of advertising, the gated recurrent unit neural networks(GRU) model is used as the ads' click through rates(CTR) predicting. Combined with the characteristics of gated unit structure and the unique of time sequence in data, using BPTT algorithm to train the model. Furthermore, by optimizing the step length algorithm of the gated unit recurrent neural networks, making the model reach optimal point better and faster in less iterative rounds. The experiment results show that the model based on the gated recurrent unit neural networks and its optimization of step length algorithm has the better effect on the ads' CTR predicting, which helps advertisers, media and audience achieve a win-win and mutually beneficial situation in Three-Side Game.

Information processing in echo state networks at the edge of chaos.

PubMed

Boedecker, Joschka; Obst, Oliver; Lizier, Joseph T; Mayer, N Michael; Asada, Minoru

2012-09-01

We investigate information processing in randomly connected recurrent neural networks. It has been shown previously that the computational capabilities of these networks are maximized when the recurrent layer is close to the border between a stable and an unstable dynamics regime, the so called edge of chaos. The reasons, however, for this maximized performance are not completely understood. We adopt an information-theoretical framework and are for the first time able to quantify the computational capabilities between elements of these networks directly as they undergo the phase transition to chaos. Specifically, we present evidence that both information transfer and storage in the recurrent layer are maximized close to this phase transition, providing an explanation for why guiding the recurrent layer toward the edge of chaos is computationally useful. As a consequence, our study suggests self-organized ways of improving performance in recurrent neural networks, driven by input data. Moreover, the networks we study share important features with biological systems such as feedback connections and online computation on input streams. A key example is the cerebral cortex, which was shown to also operate close to the edge of chaos. Consequently, the behavior of model systems as studied here is likely to shed light on reasons why biological systems are tuned into this specific regime.

Extracting recurrent scenarios from narrative texts using a Bayesian network: application to serious occupational accidents with movement disturbance.

PubMed

Abdat, F; Leclercq, S; Cuny, X; Tissot, C

2014-09-01

A probabilistic approach has been developed to extract recurrent serious Occupational Accident with Movement Disturbance (OAMD) scenarios from narrative texts within a prevention framework. Relevant data extracted from 143 accounts was initially coded as logical combinations of generic accident factors. A Bayesian Network (BN)-based model was then built for OAMDs using these data and expert knowledge. A data clustering process was subsequently performed to group the OAMDs into similar classes from generic factor occurrence and pattern standpoints. Finally, the Most Probable Explanation (MPE) was evaluated and identified as the associated recurrent scenario for each class. Using this approach, 8 scenarios were extracted to describe 143 OAMDs in the construction and metallurgy sectors. Their recurrent nature is discussed. Probable generic factor combinations provide a fair representation of particularly serious OAMDs, as described in narrative texts. This work represents a real contribution to raising company awareness of the variety of circumstances, in which these accidents occur, to progressing in the prevention of such accidents and to developing an analysis framework dedicated to this kind of accident. Copyright © 2014 Elsevier Ltd. All rights reserved.

Analysis of recurrent neural networks for short-term energy load forecasting

NASA Astrophysics Data System (ADS)

Di Persio, Luca; Honchar, Oleksandr

2017-11-01

Short-term forecasts have recently gained an increasing attention because of the rise of competitive electricity markets. In fact, short-terms forecast of possible future loads turn out to be fundamental to build efficient energy management strategies as well as to avoid energy wastage. Such type of challenges are difficult to tackle both from a theoretical and applied point of view. Latter tasks require sophisticated methods to manage multidimensional time series related to stochastic phenomena which are often highly interconnected. In the present work we first review novel approaches to energy load forecasting based on recurrent neural network, focusing our attention on long/short term memory architectures (LSTMs). Such type of artificial neural networks have been widely applied to problems dealing with sequential data such it happens, e.g., in socio-economics settings, for text recognition purposes, concerning video signals, etc., always showing their effectiveness to model complex temporal data. Moreover, we consider different novel variations of basic LSTMs, such as sequence-to-sequence approach and bidirectional LSTMs, aiming at providing effective models for energy load data. Last but not least, we test all the described algorithms on real energy load data showing not only that deep recurrent networks can be successfully applied to energy load forecasting, but also that this approach can be extended to other problems based on time series prediction.

Co-expression modules construction by WGCNA and identify potential prognostic markers of uveal melanoma.

PubMed

Wan, Qi; Tang, Jing; Han, Yu; Wang, Dan

2018-01-01

Uveal melanoma is an aggressive cancer which has a high percentage recurrence and with a worse prognosis. Identify the potential prognostic markers of uveal melanoma may provide information for early detection of recurrence and treatment. RNA sequence data of uveal melanoma and patient clinic traits were obtained from The Cancer Genome Atlas (TCGA) database. Co-expression modules were built by weighted gene co -expression network analysis (WGCNA) and applied to investigate the relationship underlying modules and clinic traits. Besides, functional enrichment analysis was performed on these co-expression genes from interested modules. First, using WGCNA, identified 21 co-expression modules were constructed by the 10975 genes from the 80 human uveal melanoma samples. The number of genes in these modules ranged from 42 to 5091. Found four co -expression modules significantly correlated with three clinic traits (status, recurrence and recurrence Time). Module red, and purple positively correlated with patient's life status and recurrence Time. Module green positively correlates with recurrence. The result of functional enrichment analysis showed that the module magenta was mainly enriched genetic material assemble processes, the purple module was mainly enriched in tissue homeostasis and melanosome membrane and the module red was mainly enriched metastasis of cell, suggesting its critical role in the recurrence and development of the disease. Additionally, identified the hug gene (top connectivity with other genes) in each module. The hub gene SLC17A7, NTRK2, ABTB1 and ADPRHL1 might play a vital role in recurrence of uveal melanoma. Our findings provided the framework of co-expression gene modules of uveal melanoma and identified some prognostic markers might be detection of recurrence and treatment for uveal melanoma. Copyright © 2017 Elsevier Ltd. All rights reserved.

Earthquake correlations and networks: A comparative study

NASA Astrophysics Data System (ADS)

Krishna Mohan, T. R.; Revathi, P. G.

2011-04-01

We quantify the correlation between earthquakes and use the same to extract causally connected earthquake pairs. Our correlation metric is a variation on the one introduced by Baiesi and Paczuski [M. Baiesi and M. Paczuski, Phys. Rev. E EULEEJ1539-375510.1103/PhysRevE.69.06610669, 066106 (2004)]. A network of earthquakes is then constructed from the time-ordered catalog and with links between the more correlated ones. A list of recurrences to each of the earthquakes is identified employing correlation thresholds to demarcate the most meaningful ones in each cluster. Data pertaining to three different seismic regions (viz., California, Japan, and the Himalayas) are comparatively analyzed using such a network model. The distribution of recurrence lengths and recurrence times are two of the key features analyzed to draw conclusions about the universal aspects of such a network model. We find that the unimodal feature of recurrence length distribution, which helps to associate typical rupture lengths with different magnitude earthquakes, is robust across the different seismic regions. The out-degree of the networks shows a hub structure rooted on the large magnitude earthquakes. In-degree distribution is seen to be dependent on the density of events in the neighborhood. Power laws, with two regimes having different exponents, are obtained with recurrence time distribution. The first regime confirms the Omori law for aftershocks while the second regime, with a faster falloff for the larger recurrence times, establishes that pure spatial recurrences also follow a power-law distribution. The crossover to the second power-law regime can be taken to be signaling the end of the aftershock regime in an objective fashion.

Predicting local field potentials with recurrent neural networks.

PubMed

Kim, Louis; Harer, Jacob; Rangamani, Akshay; Moran, James; Parks, Philip D; Widge, Alik; Eskandar, Emad; Dougherty, Darin; Chin, Sang Peter

2016-08-01

We present a Recurrent Neural Network using LSTM (Long Short Term Memory) that is capable of modeling and predicting Local Field Potentials. We train and test the network on real data recorded from epilepsy patients. We construct networks that predict multi-channel LFPs for 1, 10, and 100 milliseconds forward in time. Our results show that prediction using LSTM outperforms regression when predicting 10 and 100 millisecond forward in time.

Potential microRNA-mediated oncogenic intercellular communication revealed by pan-cancer analysis

NASA Astrophysics Data System (ADS)

Li, Yue; Zhang, Zhaolei

2014-11-01

Carcinogenesis consists of oncogenesis and metastasis, and intriguingly microRNAs (miRNAs) are involved in both processes. Although aberrant miRNA activities are prevalent in diverse tumor types, the exact mechanisms for how they regulate cancerous processes are not always clear. To this end, we performed a large-scale pan-cancer analysis via a novel probabilistic approach to infer recurrent miRNA-target interactions implicated in 12 cancer types using data from The Cancer Genome Atlas. We discovered ~20,000 recurrent miRNA regulations, which are enriched for cancer-related miRNAs/genes. Notably, miRNA 200 family (miR-200/141/429) is among the most prominent miRNA regulators, which is known to be involved in metastasis. Importantly, the recurrent miRNA regulatory network is not only enriched for cancer pathways but also for extracellular matrix (ECM) organization and ECM-receptor interactions. The results suggest an intriguing cancer mechanism involving miRNA-mediated cell-to-cell communication, which possibly involves delivery of tumorigenic miRNA messengers to adjacent cells via exosomes. Finally, survival analysis revealed 414 recurrent-prognostic associations, where both gene and miRNA involved in each interaction conferred significant prognostic power in one or more cancer types. Together, our comprehensive pan-cancer analysis provided not only biological insights into metastasis but also brought to bear the clinical relevance of the proposed recurrent miRNA-gene associations.

Short-term memory capacity in networks via the restricted isometry property.

PubMed

Charles, Adam S; Yap, Han Lun; Rozell, Christopher J

2014-06-01

Cortical networks are hypothesized to rely on transient network activity to support short-term memory (STM). In this letter, we study the capacity of randomly connected recurrent linear networks for performing STM when the input signals are approximately sparse in some basis. We leverage results from compressed sensing to provide rigorous nonasymptotic recovery guarantees, quantifying the impact of the input sparsity level, the input sparsity basis, and the network characteristics on the system capacity. Our analysis demonstrates that network memory capacities can scale superlinearly with the number of nodes and in some situations can achieve STM capacities that are much larger than the network size. We provide perfect recovery guarantees for finite sequences and recovery bounds for infinite sequences. The latter analysis predicts that network STM systems may have an optimal recovery length that balances errors due to omission and recall mistakes. Furthermore, we show that the conditions yielding optimal STM capacity can be embodied in several network topologies, including networks with sparse or dense connectivities.

Spatially dynamic recurrent information flow across long-range dorsal motor network encodes selective motor goals.

PubMed

Yoo, Peter E; Hagan, Maureen A; John, Sam E; Opie, Nicholas L; Ordidge, Roger J; O'Brien, Terence J; Oxley, Thomas J; Moffat, Bradford A; Wong, Yan T

2018-06-01

Performing voluntary movements involves many regions of the brain, but it is unknown how they work together to plan and execute specific movements. We recorded high-resolution ultra-high-field blood-oxygen-level-dependent signal during a cued ankle-dorsiflexion task. The spatiotemporal dynamics and the patterns of task-relevant information flow across the dorsal motor network were investigated. We show that task-relevant information appears and decays earlier in the higher order areas of the dorsal motor network then in the primary motor cortex. Furthermore, the results show that task-relevant information is encoded in general initially, and then selective goals are subsequently encoded in specifics subregions across the network. Importantly, the patterns of recurrent information flow across the network vary across different subregions depending on the goal. Recurrent information flow was observed across all higher order areas of the dorsal motor network in the subregions encoding for the current goal. In contrast, only the top-down information flow from the supplementary motor cortex to the frontoparietal regions, with weakened recurrent information flow between the frontoparietal regions and bottom-up information flow from the frontoparietal regions to the supplementary cortex were observed in the subregions encoding for the opposing goal. We conclude that selective motor goal encoding and execution rely on goal-dependent differences in subregional recurrent information flow patterns across the long-range dorsal motor network areas that exhibit graded functional specialization. © 2018 Wiley Periodicals, Inc.

A long non-coding RNA expression profile can predict early recurrence in hepatocellular carcinoma after curative resection.

PubMed

Lv, Yufeng; Wei, Wenhao; Huang, Zhong; Chen, Zhichao; Fang, Yuan; Pan, Lili; Han, Xueqiong; Xu, Zihai

2018-06-20

The aim of this study was to develop a novel long non-coding RNA (lncRNA) expression signature to accurately predict early recurrence for patients with hepatocellular carcinoma (HCC) after curative resection. Using expression profiles downloaded from The Cancer Genome Atlas database, we identified multiple lncRNAs with differential expression between early recurrence (ER) group and non-early recurrence (non-ER) group of HCC. Least absolute shrinkage and selection operator (LASSO) for logistic regression models were used to develop a lncRNA-based classifier for predicting ER in the training set. An independent test set was used to validated the predictive value of this classifier. Futhermore, a co-expression network based on these lncRNAs and its highly related genes was constructed and Gene Ontology and Kyoto Encyclopedia of Genes and Genomes pathway enrichment analyses of genes in the network were performed. We identified 10 differentially expressed lncRNAs, including 3 that were upregulated and 7 that were downregulated in ER group. The lncRNA-based classifier was constructed based on 7 lncRNAs (AL035661.1, PART1, AC011632.1, AC109588.1, AL365361.1, LINC00861 and LINC02084), and its accuracy was 0.83 in training set, 0.87 in test set and 0.84 in total set. And ROC curve analysis showed the AUROC was 0.741 in training set, 0.824 in the test set and 0.765 in total set. A functional enrichment analysis suggested that the genes of which is highly related to 4 lncRNAs were involved in immune system. This 7-lncRNA expression profile can effectively predict the early recurrence after surgical resection for HCC. This article is protected by copyright. All rights reserved.

Spatiotemporal discrimination in neural networks with short-term synaptic plasticity

NASA Astrophysics Data System (ADS)

Shlaer, Benjamin; Miller, Paul

2015-03-01

Cells in recurrently connected neural networks exhibit bistability, which allows for stimulus information to persist in a circuit even after stimulus offset, i.e. short-term memory. However, such a system does not have enough hysteresis to encode temporal information about the stimuli. The biophysically described phenomenon of synaptic depression decreases synaptic transmission strengths due to increased presynaptic activity. This short-term reduction in synaptic strengths can destabilize attractor states in excitatory recurrent neural networks, causing the network to move along stimulus dependent dynamical trajectories. Such a network can successfully separate amplitudes and durations of stimuli from the number of successive stimuli. Stimulus number, duration and intensity encoding in randomly connected attractor networks with synaptic depression. Front. Comput. Neurosci. 7:59., and so provides a strong candidate network for the encoding of spatiotemporal information. Here we explicitly demonstrate the capability of a recurrent neural network with short-term synaptic depression to discriminate between the temporal sequences in which spatial stimuli are presented.

Whole transcriptome RNA-Seq analysis of breast cancer recurrence risk using formalin-fixed paraffin-embedded tumor tissue.

PubMed

Sinicropi, Dominick; Qu, Kunbin; Collin, Francois; Crager, Michael; Liu, Mei-Lan; Pelham, Robert J; Pho, Mylan; Dei Rossi, Andrew; Jeong, Jennie; Scott, Aaron; Ambannavar, Ranjana; Zheng, Christina; Mena, Raul; Esteban, Jose; Stephans, James; Morlan, John; Baker, Joffre

2012-01-01

RNA biomarkers discovered by RT-PCR-based gene expression profiling of archival formalin-fixed paraffin-embedded (FFPE) tissue form the basis for widely used clinical diagnostic tests; however, RT-PCR is practically constrained in the number of transcripts that can be interrogated. We have developed and optimized RNA-Seq library chemistry as well as bioinformatics and biostatistical methods for whole transcriptome profiling from FFPE tissue. The chemistry accommodates low RNA inputs and sample multiplexing. These methods both enable rediscovery of RNA biomarkers for disease recurrence risk that were previously identified by RT-PCR analysis of a cohort of 136 patients, and also identify a high percentage of recurrence risk markers that were previously discovered using DNA microarrays in a separate cohort of patients, evidence that this RNA-Seq technology has sufficient precision and sensitivity for biomarker discovery. More than two thousand RNAs are strongly associated with breast cancer recurrence risk in the 136 patient cohort (FDR <10%). Many of these are intronic RNAs for which corresponding exons are not also associated with disease recurrence. A number of the RNAs associated with recurrence risk belong to novel RNA networks. It will be important to test the validity of these novel associations in whole transcriptome RNA-Seq screens of other breast cancer cohorts.

Whole Transcriptome RNA-Seq Analysis of Breast Cancer Recurrence Risk Using Formalin-Fixed Paraffin-Embedded Tumor Tissue

PubMed Central

Sinicropi, Dominick; Qu, Kunbin; Collin, Francois; Crager, Michael; Liu, Mei-Lan; Pelham, Robert J.; Pho, Mylan; Rossi, Andrew Dei; Jeong, Jennie; Scott, Aaron; Ambannavar, Ranjana; Zheng, Christina; Mena, Raul; Esteban, Jose; Stephans, James; Morlan, John; Baker, Joffre

2012-01-01

RNA biomarkers discovered by RT-PCR-based gene expression profiling of archival formalin-fixed paraffin-embedded (FFPE) tissue form the basis for widely used clinical diagnostic tests; however, RT-PCR is practically constrained in the number of transcripts that can be interrogated. We have developed and optimized RNA-Seq library chemistry as well as bioinformatics and biostatistical methods for whole transcriptome profiling from FFPE tissue. The chemistry accommodates low RNA inputs and sample multiplexing. These methods both enable rediscovery of RNA biomarkers for disease recurrence risk that were previously identified by RT-PCR analysis of a cohort of 136 patients, and also identify a high percentage of recurrence risk markers that were previously discovered using DNA microarrays in a separate cohort of patients, evidence that this RNA-Seq technology has sufficient precision and sensitivity for biomarker discovery. More than two thousand RNAs are strongly associated with breast cancer recurrence risk in the 136 patient cohort (FDR <10%). Many of these are intronic RNAs for which corresponding exons are not also associated with disease recurrence. A number of the RNAs associated with recurrence risk belong to novel RNA networks. It will be important to test the validity of these novel associations in whole transcriptome RNA-Seq screens of other breast cancer cohorts. PMID:22808097

A One-Layer Recurrent Neural Network for Real-Time Portfolio Optimization With Probability Criterion.

PubMed

Liu, Qingshan; Dang, Chuangyin; Huang, Tingwen

2013-02-01

This paper presents a decision-making model described by a recurrent neural network for dynamic portfolio optimization. The portfolio-optimization problem is first converted into a constrained fractional programming problem. Since the objective function in the programming problem is not convex, the traditional optimization techniques are no longer applicable for solving this problem. Fortunately, the objective function in the fractional programming is pseudoconvex on the feasible region. It leads to a one-layer recurrent neural network modeled by means of a discontinuous dynamic system. To ensure the optimal solutions for portfolio optimization, the convergence of the proposed neural network is analyzed and proved. In fact, the neural network guarantees to get the optimal solutions for portfolio-investment advice if some mild conditions are satisfied. A numerical example with simulation results substantiates the effectiveness and illustrates the characteristics of the proposed neural network.

Distributed multisensory integration in a recurrent network model through supervised learning

NASA Astrophysics Data System (ADS)

Wang, He; Wong, K. Y. Michael

Sensory integration between different modalities has been extensively studied. It is suggested that the brain integrates signals from different modalities in a Bayesian optimal way. However, how the Bayesian rule is implemented in a neural network remains under debate. In this work we propose a biologically plausible recurrent network model, which can perform Bayesian multisensory integration after trained by supervised learning. Our model is composed of two modules, each for one modality. We assume that each module is a recurrent network, whose activity represents the posterior distribution of each stimulus. The feedforward input on each module is the likelihood of each modality. Two modules are integrated through cross-links, which are feedforward connections from the other modality, and reciprocal connections, which are recurrent connections between different modules. By stochastic gradient descent, we successfully trained the feedforward and recurrent coupling matrices simultaneously, both of which resembles the Mexican-hat. We also find that there are more than one set of coupling matrices that can approximate the Bayesian theorem well. Specifically, reciprocal connections and cross-links will compensate each other if one of them is removed. Even though trained with two inputs, the network's performance with only one input is in good accordance with what is predicted by the Bayesian theorem.

Statistical downscaling of precipitation using long short-term memory recurrent neural networks

NASA Astrophysics Data System (ADS)

Misra, Saptarshi; Sarkar, Sudeshna; Mitra, Pabitra

2017-11-01

Hydrological impacts of global climate change on regional scale are generally assessed by downscaling large-scale climatic variables, simulated by General Circulation Models (GCMs), to regional, small-scale hydrometeorological variables like precipitation, temperature, etc. In this study, we propose a new statistical downscaling model based on Recurrent Neural Network with Long Short-Term Memory which captures the spatio-temporal dependencies in local rainfall. The previous studies have used several other methods such as linear regression, quantile regression, kernel regression, beta regression, and artificial neural networks. Deep neural networks and recurrent neural networks have been shown to be highly promising in modeling complex and highly non-linear relationships between input and output variables in different domains and hence we investigated their performance in the task of statistical downscaling. We have tested this model on two datasets—one on precipitation in Mahanadi basin in India and the second on precipitation in Campbell River basin in Canada. Our autoencoder coupled long short-term memory recurrent neural network model performs the best compared to other existing methods on both the datasets with respect to temporal cross-correlation, mean squared error, and capturing the extremes.

Recursive Bayesian recurrent neural networks for time-series modeling.

PubMed

Mirikitani, Derrick T; Nikolaev, Nikolay

2010-02-01

This paper develops a probabilistic approach to recursive second-order training of recurrent neural networks (RNNs) for improved time-series modeling. A general recursive Bayesian Levenberg-Marquardt algorithm is derived to sequentially update the weights and the covariance (Hessian) matrix. The main strengths of the approach are a principled handling of the regularization hyperparameters that leads to better generalization, and stable numerical performance. The framework involves the adaptation of a noise hyperparameter and local weight prior hyperparameters, which represent the noise in the data and the uncertainties in the model parameters. Experimental investigations using artificial and real-world data sets show that RNNs equipped with the proposed approach outperform standard real-time recurrent learning and extended Kalman training algorithms for recurrent networks, as well as other contemporary nonlinear neural models, on time-series modeling.

Efficient Coding and Energy Efficiency Are Promoted by Balanced Excitatory and Inhibitory Synaptic Currents in Neuronal Network

PubMed Central

Yu, Lianchun; Shen, Zhou; Wang, Chen; Yu, Yuguo

2018-01-01

Selective pressure may drive neural systems to process as much information as possible with the lowest energy cost. Recent experiment evidence revealed that the ratio between synaptic excitation and inhibition (E/I) in local cortex is generally maintained at a certain value which may influence the efficiency of energy consumption and information transmission of neural networks. To understand this issue deeply, we constructed a typical recurrent Hodgkin-Huxley network model and studied the general principles that governs the relationship among the E/I synaptic current ratio, the energy cost and total amount of information transmission. We observed in such a network that there exists an optimal E/I synaptic current ratio in the network by which the information transmission achieves the maximum with relatively low energy cost. The coding energy efficiency which is defined as the mutual information divided by the energy cost, achieved the maximum with the balanced synaptic current. Although background noise degrades information transmission and imposes an additional energy cost, we find an optimal noise intensity that yields the largest information transmission and energy efficiency at this optimal E/I synaptic transmission ratio. The maximization of energy efficiency also requires a certain part of energy cost associated with spontaneous spiking and synaptic activities. We further proved this finding with analytical solution based on the response function of bistable neurons, and demonstrated that optimal net synaptic currents are capable of maximizing both the mutual information and energy efficiency. These results revealed that the development of E/I synaptic current balance could lead a cortical network to operate at a highly efficient information transmission rate at a relatively low energy cost. The generality of neuronal models and the recurrent network configuration used here suggest that the existence of an optimal E/I cell ratio for highly efficient energy costs and information maximization is a potential principle for cortical circuit networks. Summary We conducted numerical simulations and mathematical analysis to examine the energy efficiency of neural information transmission in a recurrent network as a function of the ratio of excitatory and inhibitory synaptic connections. We obtained a general solution showing that there exists an optimal E/I synaptic ratio in a recurrent network at which the information transmission as well as the energy efficiency of this network achieves a global maximum. These results reflect general mechanisms for sensory coding processes, which may give insight into the energy efficiency of neural communication and coding. PMID:29773979

Efficient Coding and Energy Efficiency Are Promoted by Balanced Excitatory and Inhibitory Synaptic Currents in Neuronal Network.

PubMed

Yu, Lianchun; Shen, Zhou; Wang, Chen; Yu, Yuguo

2018-01-01

Selective pressure may drive neural systems to process as much information as possible with the lowest energy cost. Recent experiment evidence revealed that the ratio between synaptic excitation and inhibition (E/I) in local cortex is generally maintained at a certain value which may influence the efficiency of energy consumption and information transmission of neural networks. To understand this issue deeply, we constructed a typical recurrent Hodgkin-Huxley network model and studied the general principles that governs the relationship among the E/I synaptic current ratio, the energy cost and total amount of information transmission. We observed in such a network that there exists an optimal E/I synaptic current ratio in the network by which the information transmission achieves the maximum with relatively low energy cost. The coding energy efficiency which is defined as the mutual information divided by the energy cost, achieved the maximum with the balanced synaptic current. Although background noise degrades information transmission and imposes an additional energy cost, we find an optimal noise intensity that yields the largest information transmission and energy efficiency at this optimal E/I synaptic transmission ratio. The maximization of energy efficiency also requires a certain part of energy cost associated with spontaneous spiking and synaptic activities. We further proved this finding with analytical solution based on the response function of bistable neurons, and demonstrated that optimal net synaptic currents are capable of maximizing both the mutual information and energy efficiency. These results revealed that the development of E/I synaptic current balance could lead a cortical network to operate at a highly efficient information transmission rate at a relatively low energy cost. The generality of neuronal models and the recurrent network configuration used here suggest that the existence of an optimal E/I cell ratio for highly efficient energy costs and information maximization is a potential principle for cortical circuit networks. We conducted numerical simulations and mathematical analysis to examine the energy efficiency of neural information transmission in a recurrent network as a function of the ratio of excitatory and inhibitory synaptic connections. We obtained a general solution showing that there exists an optimal E/I synaptic ratio in a recurrent network at which the information transmission as well as the energy efficiency of this network achieves a global maximum. These results reflect general mechanisms for sensory coding processes, which may give insight into the energy efficiency of neural communication and coding.

Self-Consistent Scheme for Spike-Train Power Spectra in Heterogeneous Sparse Networks.

PubMed

Pena, Rodrigo F O; Vellmer, Sebastian; Bernardi, Davide; Roque, Antonio C; Lindner, Benjamin

2018-01-01

Recurrent networks of spiking neurons can be in an asynchronous state characterized by low or absent cross-correlations and spike statistics which resemble those of cortical neurons. Although spatial correlations are negligible in this state, neurons can show pronounced temporal correlations in their spike trains that can be quantified by the autocorrelation function or the spike-train power spectrum. Depending on cellular and network parameters, correlations display diverse patterns (ranging from simple refractory-period effects and stochastic oscillations to slow fluctuations) and it is generally not well-understood how these dependencies come about. Previous work has explored how the single-cell correlations in a homogeneous network (excitatory and inhibitory integrate-and-fire neurons with nearly balanced mean recurrent input) can be determined numerically from an iterative single-neuron simulation. Such a scheme is based on the fact that every neuron is driven by the network noise (i.e., the input currents from all its presynaptic partners) but also contributes to the network noise, leading to a self-consistency condition for the input and output spectra. Here we first extend this scheme to homogeneous networks with strong recurrent inhibition and a synaptic filter, in which instabilities of the previous scheme are avoided by an averaging procedure. We then extend the scheme to heterogeneous networks in which (i) different neural subpopulations (e.g., excitatory and inhibitory neurons) have different cellular or connectivity parameters; (ii) the number and strength of the input connections are random (Erdős-Rényi topology) and thus different among neurons. In all heterogeneous cases, neurons are lumped in different classes each of which is represented by a single neuron in the iterative scheme; in addition, we make a Gaussian approximation of the input current to the neuron. These approximations seem to be justified over a broad range of parameters as indicated by comparison with simulation results of large recurrent networks. Our method can help to elucidate how network heterogeneity shapes the asynchronous state in recurrent neural networks.

An online sleep apnea detection method based on recurrence quantification analysis.

PubMed

Nguyen, Hoa Dinh; Wilkins, Brek A; Cheng, Qi; Benjamin, Bruce Allen

2014-07-01

This paper introduces an online sleep apnea detection method based on heart rate complexity as measured by recurrence quantification analysis (RQA) statistics of heart rate variability (HRV) data. RQA statistics can capture nonlinear dynamics of a complex cardiorespiratory system during obstructive sleep apnea. In order to obtain a more robust measurement of the nonstationarity of the cardiorespiratory system, we use different fixed amount of neighbor thresholdings for recurrence plot calculation. We integrate a feature selection algorithm based on conditional mutual information to select the most informative RQA features for classification, and hence, to speed up the real-time classification process without degrading the performance of the system. Two types of binary classifiers, i.e., support vector machine and neural network, are used to differentiate apnea from normal sleep. A soft decision fusion rule is developed to combine the results of these classifiers in order to improve the classification performance of the whole system. Experimental results show that our proposed method achieves better classification results compared with the previous recurrence analysis-based approach. We also show that our method is flexible and a strong candidate for a real efficient sleep apnea detection system.

Are left ventricular ejection fraction and left atrial diameter related to atrial fibrillation recurrence after catheter ablation?

PubMed Central

Jin, Xiao; Pan, Jianke; Wu, Huanlin; Xu, Danping

2018-01-01

Abstract Atrial fibrillation (AF), the most common form of arrhythmia, is associated with the prevalence of many common cardiovascular and cerebrovascular diseases. Catheter ablation is considered the first-line therapy for AF; however, AF recurrence is very common after catheter ablation. Studies have been performed to analyze the factors associated with AF recurrence, but none have reached a consistent conclusion on whether left ventricular ejection fraction (LVEF) and left atrial diameter (LA diameter) affect AF recurrence after catheter ablation. The databases PubMed, Embase, and the Cochrane Library were used to search for relevant studies up to September 2017. RevMan 5.3.5 software provided by the Cochrane Collaboration Network was used to conduct this meta-analysis. Thirteen studies involving 2825 patients were included in this meta-analysis. Overall, the results revealed that elevated LA diameter values were significantly associated with AF recurrence in patients after catheter ablation (MD = 2.19, 95% CI: 1.63–2.75, P < .001), while baseline LVEF levels were not significantly positively associated with AF recurrence in patients after catheter ablation (MD = −0.91, 95% CI: −1.18 to 1.67, P = .14). Overall, elevated LA diameter may be associated with AF recurrence after catheter ablation; however, there was no direct relationship between LVEF values and AF recurrence after catheter ablation when baseline LVEF values are normal or mildly decreased. Besides, because of publication bias, further studies should be performed to explore the mechanisms underlying AF recurrence. PMID:29768386

Application of dynamic recurrent neural networks in nonlinear system identification

NASA Astrophysics Data System (ADS)

Du, Yun; Wu, Xueli; Sun, Huiqin; Zhang, Suying; Tian, Qiang

2006-11-01

An adaptive identification method of simple dynamic recurrent neural network (SRNN) for nonlinear dynamic systems is presented in this paper. This method based on the theory that by using the inner-states feed-back of dynamic network to describe the nonlinear kinetic characteristics of system can reflect the dynamic characteristics more directly, deduces the recursive prediction error (RPE) learning algorithm of SRNN, and improves the algorithm by studying topological structure on recursion layer without the weight values. The simulation results indicate that this kind of neural network can be used in real-time control, due to its less weight values, simpler learning algorithm, higher identification speed, and higher precision of model. It solves the problems of intricate in training algorithm and slow rate in convergence caused by the complicate topological structure in usual dynamic recurrent neural network.

A recurrent self-organizing neural fuzzy inference network.

PubMed

Juang, C F; Lin, C T

1999-01-01

A recurrent self-organizing neural fuzzy inference network (RSONFIN) is proposed in this paper. The RSONFIN is inherently a recurrent multilayered connectionist network for realizing the basic elements and functions of dynamic fuzzy inference, and may be considered to be constructed from a series of dynamic fuzzy rules. The temporal relations embedded in the network are built by adding some feedback connections representing the memory elements to a feedforward neural fuzzy network. Each weight as well as node in the RSONFIN has its own meaning and represents a special element in a fuzzy rule. There are no hidden nodes (i.e., no membership functions and fuzzy rules) initially in the RSONFIN. They are created on-line via concurrent structure identification (the construction of dynamic fuzzy if-then rules) and parameter identification (the tuning of the free parameters of membership functions). The structure learning together with the parameter learning forms a fast learning algorithm for building a small, yet powerful, dynamic neural fuzzy network. Two major characteristics of the RSONFIN can thus be seen: 1) the recurrent property of the RSONFIN makes it suitable for dealing with temporal problems and 2) no predetermination, like the number of hidden nodes, must be given, since the RSONFIN can find its optimal structure and parameters automatically and quickly. Moreover, to reduce the number of fuzzy rules generated, a flexible input partition method, the aligned clustering-based algorithm, is proposed. Various simulations on temporal problems are done and performance comparisons with some existing recurrent networks are also made. Efficiency of the RSONFIN is verified from these results.

Encoding sensory and motor patterns as time-invariant trajectories in recurrent neural networks

PubMed Central

2018-01-01

Much of the information the brain processes and stores is temporal in nature—a spoken word or a handwritten signature, for example, is defined by how it unfolds in time. However, it remains unclear how neural circuits encode complex time-varying patterns. We show that by tuning the weights of a recurrent neural network (RNN), it can recognize and then transcribe spoken digits. The model elucidates how neural dynamics in cortical networks may resolve three fundamental challenges: first, encode multiple time-varying sensory and motor patterns as stable neural trajectories; second, generalize across relevant spatial features; third, identify the same stimuli played at different speeds—we show that this temporal invariance emerges because the recurrent dynamics generate neural trajectories with appropriately modulated angular velocities. Together our results generate testable predictions as to how recurrent networks may use different mechanisms to generalize across the relevant spatial and temporal features of complex time-varying stimuli. PMID:29537963

Core reactivity estimation in space reactors using recurrent dynamic networks

NASA Technical Reports Server (NTRS)

Parlos, Alexander G.; Tsai, Wei K.

1991-01-01

A recurrent multilayer perceptron network topology is used in the identification of nonlinear dynamic systems from only the input/output measurements. The identification is performed in the discrete time domain, with the learning algorithm being a modified form of the back propagation (BP) rule. The recurrent dynamic network (RDN) developed is applied for the total core reactivity prediction of a spacecraft reactor from only neutronic power level measurements. Results indicate that the RDN can reproduce the nonlinear response of the reactor while keeping the number of nodes roughly equal to the relative order of the system. As accuracy requirements are increased, the number of required nodes also increases, however, the order of the RDN necessary to obtain such results is still in the same order of magnitude as the order of the mathematical model of the system. It is believed that use of the recurrent MLP structure with a variety of different learning algorithms may prove useful in utilizing artificial neural networks for recognition, classification, and prediction of dynamic systems.

Encoding sensory and motor patterns as time-invariant trajectories in recurrent neural networks.

PubMed

Goudar, Vishwa; Buonomano, Dean V

2018-03-14

Much of the information the brain processes and stores is temporal in nature-a spoken word or a handwritten signature, for example, is defined by how it unfolds in time. However, it remains unclear how neural circuits encode complex time-varying patterns. We show that by tuning the weights of a recurrent neural network (RNN), it can recognize and then transcribe spoken digits. The model elucidates how neural dynamics in cortical networks may resolve three fundamental challenges: first, encode multiple time-varying sensory and motor patterns as stable neural trajectories; second, generalize across relevant spatial features; third, identify the same stimuli played at different speeds-we show that this temporal invariance emerges because the recurrent dynamics generate neural trajectories with appropriately modulated angular velocities. Together our results generate testable predictions as to how recurrent networks may use different mechanisms to generalize across the relevant spatial and temporal features of complex time-varying stimuli. © 2018, Goudar et al.

Biomimetic Hybrid Feedback Feedforward Neural-Network Learning Control.

PubMed

Pan, Yongping; Yu, Haoyong

2017-06-01

This brief presents a biomimetic hybrid feedback feedforward neural-network learning control (NNLC) strategy inspired by the human motor learning control mechanism for a class of uncertain nonlinear systems. The control structure includes a proportional-derivative controller acting as a feedback servo machine and a radial-basis-function (RBF) NN acting as a feedforward predictive machine. Under the sufficient constraints on control parameters, the closed-loop system achieves semiglobal practical exponential stability, such that an accurate NN approximation is guaranteed in a local region along recurrent reference trajectories. Compared with the existing NNLC methods, the novelties of the proposed method include: 1) the implementation of an adaptive NN control to guarantee plant states being recurrent is not needed, since recurrent reference signals rather than plant states are utilized as NN inputs, which greatly simplifies the analysis and synthesis of the NNLC and 2) the domain of NN approximation can be determined a priori by the given reference signals, which leads to an easy construction of the RBF-NNs. Simulation results have verified the effectiveness of this approach.

Hierarchical singleton-type recurrent neural fuzzy networks for noisy speech recognition.

PubMed

Juang, Chia-Feng; Chiou, Chyi-Tian; Lai, Chun-Lung

2007-05-01

This paper proposes noisy speech recognition using hierarchical singleton-type recurrent neural fuzzy networks (HSRNFNs). The proposed HSRNFN is a hierarchical connection of two singleton-type recurrent neural fuzzy networks (SRNFNs), where one is used for noise filtering and the other for recognition. The SRNFN is constructed by recurrent fuzzy if-then rules with fuzzy singletons in the consequences, and their recurrent properties make them suitable for processing speech patterns with temporal characteristics. In n words recognition, n SRNFNs are created for modeling n words, where each SRNFN receives the current frame feature and predicts the next one of its modeling word. The prediction error of each SRNFN is used as recognition criterion. In filtering, one SRNFN is created, and each SRNFN recognizer is connected to the same SRNFN filter, which filters noisy speech patterns in the feature domain before feeding them to the SRNFN recognizer. Experiments with Mandarin word recognition under different types of noise are performed. Other recognizers, including multilayer perceptron (MLP), time-delay neural networks (TDNNs), and hidden Markov models (HMMs), are also tested and compared. These experiments and comparisons demonstrate good results with HSRNFN for noisy speech recognition tasks.

Identification of potential drug targets based on a computational biology algorithm for venous thromboembolism.

PubMed

Xie, Ruiqiang; Li, Lei; Chen, Lina; Li, Wan; Chen, Binbin; Jiang, Jing; Huang, Hao; Li, Yiran; He, Yuehan; Lv, Junjie; He, Weiming

2017-02-01

Venous thromboembolism (VTE) is a common, fatal and frequently recurrent disease. Changes in the activity of different coagulation factors serve as a pathophysiological basis for the recurrent risk of VTE. Systems biology approaches provide a better understanding of the pathological mechanisms responsible for recurrent VTE. In this study, a novel computational method was presented to identify the recurrent risk modules (RRMs) based on the integration of expression profiles and human signaling network, which hold promise for achieving new and deeper insights into the mechanisms responsible for VTE. The results revealed that the RRMs had good classification performance to discriminate patients with recurrent VTE. The functional annotation analysis demonstrated that the RRMs played a crucial role in the pathogenesis of VTE. Furthermore, a variety of approved drug targets in the RRM M5 were related to VTE. Thus, the M5 may be applied to select potential drug targets for combination therapy and the extended treatment of VTE.

Deep Gate Recurrent Neural Network

DTIC Science & Technology

2016-11-22

Schmidhuber. A system for robotic heart surgery that learns to tie knots using recurrent neural networks. In IEEE International Conference on...tasks, such as Machine Translation (Bahdanau et al. (2015)) or Robot Reinforcement Learning (Bakker (2001)). The main idea behind these networks is to...and J. Peters. Reinforcement learning in robotics : A survey. The International Journal of Robotics Research, 32:1238–1274, 2013. ISSN 0278-3649. doi

The default mode network and recurrent depression: a neurobiological model of cognitive risk factors.

PubMed

Marchetti, Igor; Koster, Ernst H W; Sonuga-Barke, Edmund J; De Raedt, Rudi

2012-09-01

A neurobiological account of cognitive vulnerability for recurrent depression is presented based on recent developments of resting state neural networks. We propose that alterations in the interplay between task positive (TP) and task negative (TN) elements of the Default Mode Network (DMN) act as a neurobiological risk factor for recurrent depression mediated by cognitive mechanisms. In the framework, depression is characterized by an imbalance between TN-TP components leading to an overpowering of TP by TN activity. The TN-TP imbalance is associated with a dysfunctional internally-focused cognitive style as well as a failure to attenuate TN activity in the transition from rest to task. Thus we propose the TN-TP imbalance as overarching neural mechanism involved in crucial cognitive risk factors for recurrent depression, namely rumination, impaired attentional control, and cognitive reactivity. During remission the TN-TP imbalance persists predisposing to vulnerability of recurrent depression. Empirical data to support this model is reviewed. Finally, we specify how this framework can guide future research efforts.

Network public opinion space sentiment tendency analyze based on recurrent convolution neural network

NASA Astrophysics Data System (ADS)

Zhang, Gaowei; Xu, Lingyu; Wang, Lei

2018-04-01

The purpose of this chapter is to analyze the investor's psychological characteristics and investment decision-making behavior characteristics, to study the investor sentiment under the network public opinion, and then analyze from three aspects: First, investor sentiment analysis and how to spread in the online media; The influence mechanism of investor's emotion on the stock market and its effect; the third one is to measure the investor's emotion based on the degree of attention, trying hard to sort out the internal relations between the investor's sentiment and the network public opinion and the stock market, in order to lay the theoretical foundation of this article.

Different-Level Simultaneous Minimization Scheme for Fault Tolerance of Redundant Manipulator Aided with Discrete-Time Recurrent Neural Network

PubMed Central

Jin, Long; Liao, Bolin; Liu, Mei; Xiao, Lin; Guo, Dongsheng; Yan, Xiaogang

2017-01-01

By incorporating the physical constraints in joint space, a different-level simultaneous minimization scheme, which takes both the robot kinematics and robot dynamics into account, is presented and investigated for fault-tolerant motion planning of redundant manipulator in this paper. The scheme is reformulated as a quadratic program (QP) with equality and bound constraints, which is then solved by a discrete-time recurrent neural network. Simulative verifications based on a six-link planar redundant robot manipulator substantiate the efficacy and accuracy of the presented acceleration fault-tolerant scheme, the resultant QP and the corresponding discrete-time recurrent neural network. PMID:28955217

Training trajectories by continuous recurrent multilayer networks.

PubMed

Leistritz, L; Galicki, M; Witte, H; Kochs, E

2002-01-01

This paper addresses the problem of training trajectories by means of continuous recurrent neural networks whose feedforward parts are multilayer perceptrons. Such networks can approximate a general nonlinear dynamic system with arbitrary accuracy. The learning process is transformed into an optimal control framework where the weights are the controls to be determined. A training algorithm based upon a variational formulation of Pontryagin's maximum principle is proposed for such networks. Computer examples demonstrating the efficiency of the given approach are also presented.

Dopamine-Modulated Recurrent Corticoefferent Feedback in Primary Sensory Cortex Promotes Detection of Behaviorally Relevant Stimuli

PubMed Central

Handschuh, Juliane

2014-01-01

Dopaminergic neurotransmission in primary auditory cortex (AI) has been shown to be involved in learning and memory functions. Moreover, dopaminergic projections and D1/D5 receptor distributions display a layer-dependent organization, suggesting specific functions in the cortical circuitry. However, the circuit effects of dopaminergic neurotransmission in sensory cortex and their possible roles in perception, learning, and memory are largely unknown. Here, we investigated layer-specific circuit effects of dopaminergic neuromodulation using current source density (CSD) analysis in AI of Mongolian gerbils. Pharmacological stimulation of D1/D5 receptors increased auditory-evoked synaptic currents in infragranular layers, prolonging local thalamocortical input via positive feedback between infragranular output and granular input. Subsequently, dopamine promoted sustained cortical activation by prolonged recruitment of long-range corticocortical networks. A detailed circuit analysis combining layer-specific intracortical microstimulation (ICMS), CSD analysis, and pharmacological cortical silencing revealed that cross-laminar feedback enhanced by dopamine relied on a positive, fast-acting recurrent corticoefferent loop, most likely relayed via local thalamic circuits. Behavioral signal detection analysis further showed that activation of corticoefferent output by infragranular ICMS, which mimicked auditory activation under dopaminergic influence, was most effective in eliciting a behaviorally detectable signal. Our results show that D1/D5-mediated dopaminergic modulation in sensory cortex regulates positive recurrent corticoefferent feedback, which enhances states of high, persistent activity in sensory cortex evoked by behaviorally relevant stimuli. In boosting horizontal network interactions, this potentially promotes the readout of task-related information from cortical synapses and improves behavioral stimulus detection. PMID:24453315

A recurrence network approach to analyzing forced synchronization in hydrodynamic systems

NASA Astrophysics Data System (ADS)

Murugesan, Meenatchidevi; Zhu, Yuanhang; Li, Larry K. B.

2016-11-01

Hydrodynamically self-excited systems can lock into external forcing, but their lock-in boundaries and the specific bifurcations through which they lock in can be difficult to detect. We propose using recurrence networks to analyze forced synchronization in a hydrodynamic system: a low-density jet. We find that as the jet bifurcates from periodicity (unforced) to quasiperiodicity (weak forcing) and then to lock-in (strong forcing), its recurrence network changes from a regular distribution of links between nodes (unforced) to a disordered topology (weak forcing) and then to a regular distribution again at lock-in (strong forcing). The emergence of order at lock-in can be either smooth or abrupt depending on the specific lock-in route taken. Furthermore, we find that before lock-in, the probability distribution of links in the network is a function of the characteristic scales of the system, which can be quantified with network measures and used to estimate the proximity to the lock-in boundaries. This study shows that recurrence networks can be used (i) to detect lock-in, (ii) to distinguish between different routes to lock-in, and (iii) as an early warning indicator of the proximity of a system to its lock-in boundaries. This work was supported by the Research Grants Council of Hong Kong (Project No. 16235716 and 26202815).

A generalized LSTM-like training algorithm for second-order recurrent neural networks

PubMed Central

Monner, Derek; Reggia, James A.

2011-01-01

The Long Short Term Memory (LSTM) is a second-order recurrent neural network architecture that excels at storing sequential short-term memories and retrieving them many time-steps later. LSTM’s original training algorithm provides the important properties of spatial and temporal locality, which are missing from other training approaches, at the cost of limiting it’s applicability to a small set of network architectures. Here we introduce the Generalized Long Short-Term Memory (LSTM-g) training algorithm, which provides LSTM-like locality while being applicable without modification to a much wider range of second-order network architectures. With LSTM-g, all units have an identical set of operating instructions for both activation and learning, subject only to the configuration of their local environment in the network; this is in contrast to the original LSTM training algorithm, where each type of unit has its own activation and training instructions. When applied to LSTM architectures with peephole connections, LSTM-g takes advantage of an additional source of back-propagated error which can enable better performance than the original algorithm. Enabled by the broad architectural applicability of LSTM-g, we demonstrate that training recurrent networks engineered for specific tasks can produce better results than single-layer networks. We conclude that LSTM-g has the potential to both improve the performance and broaden the applicability of spatially and temporally local gradient-based training algorithms for recurrent neural networks. PMID:21803542

Detecting recurrent gene mutation in interaction network context using multi-scale graph diffusion.

PubMed

Babaei, Sepideh; Hulsman, Marc; Reinders, Marcel; de Ridder, Jeroen

2013-01-23

Delineating the molecular drivers of cancer, i.e. determining cancer genes and the pathways which they deregulate, is an important challenge in cancer research. In this study, we aim to identify pathways of frequently mutated genes by exploiting their network neighborhood encoded in the protein-protein interaction network. To this end, we introduce a multi-scale diffusion kernel and apply it to a large collection of murine retroviral insertional mutagenesis data. The diffusion strength plays the role of scale parameter, determining the size of the network neighborhood that is taken into account. As a result, in addition to detecting genes with frequent mutations in their genomic vicinity, we find genes that harbor frequent mutations in their interaction network context. We identify densely connected components of known and putatively novel cancer genes and demonstrate that they are strongly enriched for cancer related pathways across the diffusion scales. Moreover, the mutations in the clusters exhibit a significant pattern of mutual exclusion, supporting the conjecture that such genes are functionally linked. Using multi-scale diffusion kernel, various infrequently mutated genes are found to harbor significant numbers of mutations in their interaction network neighborhood. Many of them are well-known cancer genes. The results demonstrate the importance of defining recurrent mutations while taking into account the interaction network context. Importantly, the putative cancer genes and networks detected in this study are found to be significant at different diffusion scales, confirming the necessity of a multi-scale analysis.

Segmented-memory recurrent neural networks.

PubMed

Chen, Jinmiao; Chaudhari, Narendra S

2009-08-01

Conventional recurrent neural networks (RNNs) have difficulties in learning long-term dependencies. To tackle this problem, we propose an architecture called segmented-memory recurrent neural network (SMRNN). A symbolic sequence is broken into segments and then presented as inputs to the SMRNN one symbol per cycle. The SMRNN uses separate internal states to store symbol-level context, as well as segment-level context. The symbol-level context is updated for each symbol presented for input. The segment-level context is updated after each segment. The SMRNN is trained using an extended real-time recurrent learning algorithm. We test the performance of SMRNN on the information latching problem, the "two-sequence problem" and the problem of protein secondary structure (PSS) prediction. Our implementation results indicate that SMRNN performs better on long-term dependency problems than conventional RNNs. Besides, we also theoretically analyze how the segmented memory of SMRNN helps learning long-term temporal dependencies and study the impact of the segment length.

Application of Deep Learning of Multi-Temporal SENTINEL-1 Images for the Classification of Coastal Vegetation Zone of the Danube Delta

NASA Astrophysics Data System (ADS)

Niculescu, S.; Ienco, D.; Hanganu, J.

2018-04-01

Land cover is a fundamental variable for regional planning, as well as for the study and understanding of the environment. This work propose a multi-temporal approach relying on a fusion of radar multi-sensor data and information collected by the latest sensor (Sentinel-1) with a view to obtaining better results than traditional image processing techniques. The Danube Delta is the site for this work. The spatial approach relies on new spatial analysis technologies and methodologies: Deep Learning of multi-temporal Sentinel-1. We propose a deep learning network for image classification which exploits the multi-temporal characteristic of Sentinel-1 data. The model we employ is a Gated Recurrent Unit (GRU) Network, a recurrent neural network that explicitly takes into account the time dimension via a gated mechanism to perform the final prediction. The main quality of the GRU network is its ability to consider only the important part of the information coming from the temporal data discarding the irrelevant information via a forgetting mechanism. We propose to use such network structure to classify a series of images Sentinel-1 (20 Sentinel-1 images acquired between 9.10.2014 and 01.04.2016). The results are compared with results of the classification of Random Forest.

Spatial Learning and Action Planning in a Prefrontal Cortical Network Model

PubMed Central

Martinet, Louis-Emmanuel; Sheynikhovich, Denis; Benchenane, Karim; Arleo, Angelo

2011-01-01

The interplay between hippocampus and prefrontal cortex (PFC) is fundamental to spatial cognition. Complementing hippocampal place coding, prefrontal representations provide more abstract and hierarchically organized memories suitable for decision making. We model a prefrontal network mediating distributed information processing for spatial learning and action planning. Specific connectivity and synaptic adaptation principles shape the recurrent dynamics of the network arranged in cortical minicolumns. We show how the PFC columnar organization is suitable for learning sparse topological-metrical representations from redundant hippocampal inputs. The recurrent nature of the network supports multilevel spatial processing, allowing structural features of the environment to be encoded. An activation diffusion mechanism spreads the neural activity through the column population leading to trajectory planning. The model provides a functional framework for interpreting the activity of PFC neurons recorded during navigation tasks. We illustrate the link from single unit activity to behavioral responses. The results suggest plausible neural mechanisms subserving the cognitive “insight” capability originally attributed to rodents by Tolman & Honzik. Our time course analysis of neural responses shows how the interaction between hippocampus and PFC can yield the encoding of manifold information pertinent to spatial planning, including prospective coding and distance-to-goal correlates. PMID:21625569

Self-Consistent Scheme for Spike-Train Power Spectra in Heterogeneous Sparse Networks

PubMed Central

Pena, Rodrigo F. O.; Vellmer, Sebastian; Bernardi, Davide; Roque, Antonio C.; Lindner, Benjamin

2018-01-01

Recurrent networks of spiking neurons can be in an asynchronous state characterized by low or absent cross-correlations and spike statistics which resemble those of cortical neurons. Although spatial correlations are negligible in this state, neurons can show pronounced temporal correlations in their spike trains that can be quantified by the autocorrelation function or the spike-train power spectrum. Depending on cellular and network parameters, correlations display diverse patterns (ranging from simple refractory-period effects and stochastic oscillations to slow fluctuations) and it is generally not well-understood how these dependencies come about. Previous work has explored how the single-cell correlations in a homogeneous network (excitatory and inhibitory integrate-and-fire neurons with nearly balanced mean recurrent input) can be determined numerically from an iterative single-neuron simulation. Such a scheme is based on the fact that every neuron is driven by the network noise (i.e., the input currents from all its presynaptic partners) but also contributes to the network noise, leading to a self-consistency condition for the input and output spectra. Here we first extend this scheme to homogeneous networks with strong recurrent inhibition and a synaptic filter, in which instabilities of the previous scheme are avoided by an averaging procedure. We then extend the scheme to heterogeneous networks in which (i) different neural subpopulations (e.g., excitatory and inhibitory neurons) have different cellular or connectivity parameters; (ii) the number and strength of the input connections are random (Erdős-Rényi topology) and thus different among neurons. In all heterogeneous cases, neurons are lumped in different classes each of which is represented by a single neuron in the iterative scheme; in addition, we make a Gaussian approximation of the input current to the neuron. These approximations seem to be justified over a broad range of parameters as indicated by comparison with simulation results of large recurrent networks. Our method can help to elucidate how network heterogeneity shapes the asynchronous state in recurrent neural networks. PMID:29551968

Magnetic resonance analysis of malignant transformation in recurrent glioma.

PubMed

Jalbert, Llewellyn E; Neill, Evan; Phillips, Joanna J; Lupo, Janine M; Olson, Marram P; Molinaro, Annette M; Berger, Mitchel S; Chang, Susan M; Nelson, Sarah J

2016-08-01

Patients with low-grade glioma (LGG) have a relatively long survival, and a balance is often struck between treating the tumor and impacting quality of life. While lesions may remain stable for many years, they may also undergo malignant transformation (MT) at the time of recurrence and require more aggressive intervention. Here we report on a state-of-the-art multiparametric MRI study of patients with recurrent LGG. One hundred and eleven patients previously diagnosed with LGG were scanned at either 1.5 T or 3 T MR at the time of recurrence. Volumetric and intensity parameters were estimated from anatomic, diffusion, perfusion, and metabolic MR data. Direct comparisons of histopathological markers from image-guided tissue samples with metrics derived from the corresponding locations on the in vivo images were made. A bioinformatics approach was applied to visualize and interpret these results, which included imaging heatmaps and network analysis. Multivariate linear-regression modeling was utilized for predicting transformation. Many advanced imaging parameters were found to be significantly different for patients with tumors that had undergone MT versus those that had not. Imaging metrics calculated at the tissue sample locations highlighted the distinct biological significance of the imaging and the heterogeneity present in recurrent LGG, while multivariate modeling yielded a 76.04% accuracy in predicting MT. The acquisition and quantitative analysis of such multiparametric MR data may ultimately allow for improved clinical assessment and treatment stratification for patients with recurrent LGG. © The Author(s) 2016. Published by Oxford University Press on behalf of the Society for Neuro-Oncology.

Engine cylinder pressure reconstruction using crank kinematics and recurrently-trained neural networks

NASA Astrophysics Data System (ADS)

Bennett, C.; Dunne, J. F.; Trimby, S.; Richardson, D.

2017-02-01

A recurrent non-linear autoregressive with exogenous input (NARX) neural network is proposed, and a suitable fully-recurrent training methodology is adapted and tuned, for reconstructing cylinder pressure in multi-cylinder IC engines using measured crank kinematics. This type of indirect sensing is important for cost effective closed-loop combustion control and for On-Board Diagnostics. The challenge addressed is to accurately predict cylinder pressure traces within the cycle under generalisation conditions: i.e. using data not previously seen by the network during training. This involves direct construction and calibration of a suitable inverse crank dynamic model, which owing to singular behaviour at top-dead-centre (TDC), has proved difficult via physical model construction, calibration, and inversion. The NARX architecture is specialised and adapted to cylinder pressure reconstruction, using a fully-recurrent training methodology which is needed because the alternatives are too slow and unreliable for practical network training on production engines. The fully-recurrent Robust Adaptive Gradient Descent (RAGD) algorithm, is tuned initially using synthesised crank kinematics, and then tested on real engine data to assess the reconstruction capability. Real data is obtained from a 1.125 l, 3-cylinder, in-line, direct injection spark ignition (DISI) engine involving synchronised measurements of crank kinematics and cylinder pressure across a range of steady-state speed and load conditions. The paper shows that a RAGD-trained NARX network using both crank velocity and crank acceleration as input information, provides fast and robust training. By using the optimum epoch identified during RAGD training, acceptably accurate cylinder pressures, and especially accurate location-of-peak-pressure, can be reconstructed robustly under generalisation conditions, making it the most practical NARX configuration and recurrent training methodology for use on production engines.

Diagonal recurrent neural network based adaptive control of nonlinear dynamical systems using lyapunov stability criterion.

PubMed

Kumar, Rajesh; Srivastava, Smriti; Gupta, J R P

2017-03-01

In this paper adaptive control of nonlinear dynamical systems using diagonal recurrent neural network (DRNN) is proposed. The structure of DRNN is a modification of fully connected recurrent neural network (FCRNN). Presence of self-recurrent neurons in the hidden layer of DRNN gives it an ability to capture the dynamic behaviour of the nonlinear plant under consideration (to be controlled). To ensure stability, update rules are developed using lyapunov stability criterion. These rules are then used for adjusting the various parameters of DRNN. The responses of plants obtained with DRNN are compared with those obtained when multi-layer feed forward neural network (MLFFNN) is used as a controller. Also, in example 4, FCRNN is also investigated and compared with DRNN and MLFFNN. Robustness of the proposed control scheme is also tested against parameter variations and disturbance signals. Four simulation examples including one-link robotic manipulator and inverted pendulum are considered on which the proposed controller is applied. The results so obtained show the superiority of DRNN over MLFFNN as a controller. Copyright © 2017 ISA. Published by Elsevier Ltd. All rights reserved.

Mining for recurrent long-range interactions in RNA structures reveals embedded hierarchies in network families.

PubMed

Reinharz, Vladimir; Soulé, Antoine; Westhof, Eric; Waldispühl, Jérôme; Denise, Alain

2018-05-04

The wealth of the combinatorics of nucleotide base pairs enables RNA molecules to assemble into sophisticated interaction networks, which are used to create complex 3D substructures. These interaction networks are essential to shape the 3D architecture of the molecule, and also to provide the key elements to carry molecular functions such as protein or ligand binding. They are made of organised sets of long-range tertiary interactions which connect distinct secondary structure elements in 3D structures. Here, we present a de novo data-driven approach to extract automatically from large data sets of full RNA 3D structures the recurrent interaction networks (RINs). Our methodology enables us for the first time to detect the interaction networks connecting distinct components of the RNA structure, highlighting their diversity and conservation through non-related functional RNAs. We use a graphical model to perform pairwise comparisons of all RNA structures available and to extract RINs and modules. Our analysis yields a complete catalog of RNA 3D structures available in the Protein Data Bank and reveals the intricate hierarchical organization of the RNA interaction networks and modules. We assembled our results in an online database (http://carnaval.lri.fr) which will be regularly updated. Within the site, a tool allows users with a novel RNA structure to detect automatically whether the novel structure contains previously observed RINs.

Emergent latent symbol systems in recurrent neural networks

NASA Astrophysics Data System (ADS)

Monner, Derek; Reggia, James A.

2012-12-01

Fodor and Pylyshyn [(1988). Connectionism and cognitive architecture: A critical analysis. Cognition, 28(1-2), 3-71] famously argued that neural networks cannot behave systematically short of implementing a combinatorial symbol system. A recent response from Frank et al. [(2009). Connectionist semantic systematicity. Cognition, 110(3), 358-379] claimed to have trained a neural network to behave systematically without implementing a symbol system and without any in-built predisposition towards combinatorial representations. We believe systems like theirs may in fact implement a symbol system on a deeper and more interesting level: one where the symbols are latent - not visible at the level of network structure. In order to illustrate this possibility, we demonstrate our own recurrent neural network that learns to understand sentence-level language in terms of a scene. We demonstrate our model's learned understanding by testing it on novel sentences and scenes. By paring down our model into an architecturally minimal version, we demonstrate how it supports combinatorial computation over distributed representations by using the associative memory operations of Vector Symbolic Architectures. Knowledge of the model's memory scheme gives us tools to explain its errors and construct superior future models. We show how the model designs and manipulates a latent symbol system in which the combinatorial symbols are patterns of activation distributed across the layers of a neural network, instantiating a hybrid of classical symbolic and connectionist representations that combines advantages of both.

Adaptive dynamic surface control of flexible-joint robots using self-recurrent wavelet neural networks.

PubMed

Yoo, Sung Jin; Park, Jin Bae; Choi, Yoon Ho

2006-12-01

A new method for the robust control of flexible-joint (FJ) robots with model uncertainties in both robot dynamics and actuator dynamics is proposed. The proposed control system is a combination of the adaptive dynamic surface control (DSC) technique and the self-recurrent wavelet neural network (SRWNN). The adaptive DSC technique provides the ability to overcome the "explosion of complexity" problem in backstepping controllers. The SRWNNs are used to observe the arbitrary model uncertainties of FJ robots, and all their weights are trained online. From the Lyapunov stability analysis, their adaptation laws are induced, and the uniformly ultimately boundedness of all signals in a closed-loop adaptive system is proved. Finally, simulation results for a three-link FJ robot are utilized to validate the good position tracking performance and robustness against payload uncertainties and external disturbances of the proposed control system.

Scale-Limited Lagrange Stability and Finite-Time Synchronization for Memristive Recurrent Neural Networks on Time Scales.

PubMed

Xiao, Qiang; Zeng, Zhigang

2017-10-01

The existed results of Lagrange stability and finite-time synchronization for memristive recurrent neural networks (MRNNs) are scale-free on time evolvement, and some restrictions appear naturally. In this paper, two novel scale-limited comparison principles are established by means of inequality techniques and induction principle on time scales. Then the results concerning Lagrange stability and global finite-time synchronization of MRNNs on time scales are obtained. Scaled-limited Lagrange stability criteria are derived, in detail, via nonsmooth analysis and theory of time scales. Moreover, novel criteria for achieving the global finite-time synchronization are acquired. In addition, the derived method can also be used to study global finite-time stabilization. The proposed results extend or improve the existed ones in the literatures. Two numerical examples are chosen to show the effectiveness of the obtained results.

Training the Recurrent neural network by the Fuzzy Min-Max algorithm for fault prediction

DOE Office of Scientific and Technical Information (OSTI.GOV)

Zemouri, Ryad; Racoceanu, Daniel; Zerhouni, Noureddine

2009-03-05

In this paper, we present a training technique of a Recurrent Radial Basis Function neural network for fault prediction. We use the Fuzzy Min-Max technique to initialize the k-center of the RRBF neural network. The k-means algorithm is then applied to calculate the centers that minimize the mean square error of the prediction task. The performances of the k-means algorithm are then boosted by the Fuzzy Min-Max technique.

Network perturbation by recurrent regulatory variants in cancer

PubMed Central

Cho, Ara; Lee, Insuk; Choi, Jung Kyoon

2017-01-01

Cancer driving genes have been identified as recurrently affected by variants that alter protein-coding sequences. However, a majority of cancer variants arise in noncoding regions, and some of them are thought to play a critical role through transcriptional perturbation. Here we identified putative transcriptional driver genes based on combinatorial variant recurrence in cis-regulatory regions. The identified genes showed high connectivity in the cancer type-specific transcription regulatory network, with high outdegree and many downstream genes, highlighting their causative role during tumorigenesis. In the protein interactome, the identified transcriptional drivers were not as highly connected as coding driver genes but appeared to form a network module centered on the coding drivers. The coding and regulatory variants associated via these interactions between the coding and transcriptional drivers showed exclusive and complementary occurrence patterns across tumor samples. Transcriptional cancer drivers may act through an extensive perturbation of the regulatory network and by altering protein network modules through interactions with coding driver genes. PMID:28333928

Computational Modeling of Statistical Learning: Effects of Transitional Probability versus Frequency and Links to Word Learning

ERIC Educational Resources Information Center

Mirman, Daniel; Estes, Katharine Graf; Magnuson, James S.

2010-01-01

Statistical learning mechanisms play an important role in theories of language acquisition and processing. Recurrent neural network models have provided important insights into how these mechanisms might operate. We examined whether such networks capture two key findings in human statistical learning. In Simulation 1, a simple recurrent network…

Global exponential periodicity and stability of discrete-time complex-valued recurrent neural networks with time-delays.

PubMed

Hu, Jin; Wang, Jun

2015-06-01

In recent years, complex-valued recurrent neural networks have been developed and analysed in-depth in view of that they have good modelling performance for some applications involving complex-valued elements. In implementing continuous-time dynamical systems for simulation or computational purposes, it is quite necessary to utilize a discrete-time model which is an analogue of the continuous-time system. In this paper, we analyse a discrete-time complex-valued recurrent neural network model and obtain the sufficient conditions on its global exponential periodicity and exponential stability. Simulation results of several numerical examples are delineated to illustrate the theoretical results and an application on associative memory is also given. Copyright © 2015 Elsevier Ltd. All rights reserved.

Recurrent Neural Networks for Multivariate Time Series with Missing Values.

PubMed

Che, Zhengping; Purushotham, Sanjay; Cho, Kyunghyun; Sontag, David; Liu, Yan

2018-04-17

Multivariate time series data in practical applications, such as health care, geoscience, and biology, are characterized by a variety of missing values. In time series prediction and other related tasks, it has been noted that missing values and their missing patterns are often correlated with the target labels, a.k.a., informative missingness. There is very limited work on exploiting the missing patterns for effective imputation and improving prediction performance. In this paper, we develop novel deep learning models, namely GRU-D, as one of the early attempts. GRU-D is based on Gated Recurrent Unit (GRU), a state-of-the-art recurrent neural network. It takes two representations of missing patterns, i.e., masking and time interval, and effectively incorporates them into a deep model architecture so that it not only captures the long-term temporal dependencies in time series, but also utilizes the missing patterns to achieve better prediction results. Experiments of time series classification tasks on real-world clinical datasets (MIMIC-III, PhysioNet) and synthetic datasets demonstrate that our models achieve state-of-the-art performance and provide useful insights for better understanding and utilization of missing values in time series analysis.

Systematic review and network meta-analysis comparing clinical outcomes and effectiveness of surgical treatments for haemorrhoids.

PubMed

Simillis, C; Thoukididou, S N; Slesser, A A P; Rasheed, S; Tan, E; Tekkis, P P

2015-12-01

The aim was to compare the clinical outcomes and effectiveness of surgical treatments for haemorrhoids. Randomized clinical trials were identified by means of a systematic review. A Bayesian network meta-analysis was performed using the Markov chain Monte Carlo method in WinBUGS. Ninety-eight trials were included with 7827 participants and 11 surgical treatments for grade III and IV haemorrhoids. Open, closed and radiofrequency haemorrhoidectomies resulted in significantly more postoperative complications than transanal haemorrhoidal dearterialization (THD), LigaSure™ and Harmonic® haemorrhoidectomies. THD had significantly less postoperative bleeding than open and stapled procedures, and resulted in significantly fewer emergency reoperations than open, closed, stapled and LigaSure™ haemorrhoidectomies. Open and closed haemorrhoidectomies resulted in more pain on postoperative day 1 than stapled, THD, LigaSure™ and Harmonic® procedures. After stapled, LigaSure™ and Harmonic® haemorrhoidectomies patients resumed normal daily activities earlier than after open and closed procedures. THD provided the earliest time to first bowel movement. The stapled and THD groups had significantly higher haemorrhoid recurrence rates than the open, closed and LigaSure™ groups. Recurrence of haemorrhoidal symptoms was more common after stapled haemorrhoidectomy than after open and LigaSure™ operations. No significant difference was identified between treatments for anal stenosis, incontinence and perianal skin tags. Open and closed haemorrhoidectomies resulted in more postoperative complications and slower recovery, but fewer haemorrhoid recurrences. THD and stapled haemorrhoidectomies were associated with decreased postoperative pain and faster recovery, but higher recurrence rates. The advantages and disadvantages of each surgical treatment should be discussed with the patient before surgery to allow an informed decision to be made. © 2015 BJS Society Ltd Published by John Wiley & Sons Ltd.

Recurrent Neural Network for Computing the Drazin Inverse.

PubMed

Stanimirović, Predrag S; Zivković, Ivan S; Wei, Yimin

2015-11-01

This paper presents a recurrent neural network (RNN) for computing the Drazin inverse of a real matrix in real time. This recurrent neural network (RNN) is composed of n independent parts (subnetworks), where n is the order of the input matrix. These subnetworks can operate concurrently, so parallel and distributed processing can be achieved. In this way, the computational advantages over the existing sequential algorithms can be attained in real-time applications. The RNN defined in this paper is convenient for an implementation in an electronic circuit. The number of neurons in the neural network is the same as the number of elements in the output matrix, which represents the Drazin inverse. The difference between the proposed RNN and the existing ones for the Drazin inverse computation lies in their network architecture and dynamics. The conditions that ensure the stability of the defined RNN as well as its convergence toward the Drazin inverse are considered. In addition, illustrative examples and examples of application to the practical engineering problems are discussed to show the efficacy of the proposed neural network.

Memory replay in balanced recurrent networks

PubMed Central

Chenkov, Nikolay; Sprekeler, Henning; Kempter, Richard

2017-01-01

Complex patterns of neural activity appear during up-states in the neocortex and sharp waves in the hippocampus, including sequences that resemble those during prior behavioral experience. The mechanisms underlying this replay are not well understood. How can small synaptic footprints engraved by experience control large-scale network activity during memory retrieval and consolidation? We hypothesize that sparse and weak synaptic connectivity between Hebbian assemblies are boosted by pre-existing recurrent connectivity within them. To investigate this idea, we connect sequences of assemblies in randomly connected spiking neuronal networks with a balance of excitation and inhibition. Simulations and analytical calculations show that recurrent connections within assemblies allow for a fast amplification of signals that indeed reduces the required number of inter-assembly connections. Replay can be evoked by small sensory-like cues or emerge spontaneously by activity fluctuations. Global—potentially neuromodulatory—alterations of neuronal excitability can switch between network states that favor retrieval and consolidation. PMID:28135266

Back-propagation learning of infinite-dimensional dynamical systems.

PubMed

Tokuda, Isao; Tokunaga, Ryuji; Aihara, Kazuyuki

2003-10-01

This paper presents numerical studies of applying back-propagation learning to a delayed recurrent neural network (DRNN). The DRNN is a continuous-time recurrent neural network having time delayed feedbacks and the back-propagation learning is to teach spatio-temporal dynamics to the DRNN. Since the time-delays make the dynamics of the DRNN infinite-dimensional, the learning algorithm and the learning capability of the DRNN are different from those of the ordinary recurrent neural network (ORNN) having no time-delays. First, two types of learning algorithms are developed for a class of DRNNs. Then, using chaotic signals generated from the Mackey-Glass equation and the Rössler equations, learning capability of the DRNN is examined. Comparing the learning algorithms, learning capability, and robustness against noise of the DRNN with those of the ORNN and time delay neural network, advantages as well as disadvantages of the DRNN are investigated.

Spatiotemporal Recurrent Convolutional Networks for Traffic Prediction in Transportation Networks

PubMed Central

Yu, Haiyang; Wu, Zhihai; Wang, Shuqin; Wang, Yunpeng; Ma, Xiaolei

2017-01-01

Predicting large-scale transportation network traffic has become an important and challenging topic in recent decades. Inspired by the domain knowledge of motion prediction, in which the future motion of an object can be predicted based on previous scenes, we propose a network grid representation method that can retain the fine-scale structure of a transportation network. Network-wide traffic speeds are converted into a series of static images and input into a novel deep architecture, namely, spatiotemporal recurrent convolutional networks (SRCNs), for traffic forecasting. The proposed SRCNs inherit the advantages of deep convolutional neural networks (DCNNs) and long short-term memory (LSTM) neural networks. The spatial dependencies of network-wide traffic can be captured by DCNNs, and the temporal dynamics can be learned by LSTMs. An experiment on a Beijing transportation network with 278 links demonstrates that SRCNs outperform other deep learning-based algorithms in both short-term and long-term traffic prediction. PMID:28672867

Spatiotemporal Recurrent Convolutional Networks for Traffic Prediction in Transportation Networks.

PubMed

Yu, Haiyang; Wu, Zhihai; Wang, Shuqin; Wang, Yunpeng; Ma, Xiaolei

2017-06-26

Predicting large-scale transportation network traffic has become an important and challenging topic in recent decades. Inspired by the domain knowledge of motion prediction, in which the future motion of an object can be predicted based on previous scenes, we propose a network grid representation method that can retain the fine-scale structure of a transportation network. Network-wide traffic speeds are converted into a series of static images and input into a novel deep architecture, namely, spatiotemporal recurrent convolutional networks (SRCNs), for traffic forecasting. The proposed SRCNs inherit the advantages of deep convolutional neural networks (DCNNs) and long short-term memory (LSTM) neural networks. The spatial dependencies of network-wide traffic can be captured by DCNNs, and the temporal dynamics can be learned by LSTMs. An experiment on a Beijing transportation network with 278 links demonstrates that SRCNs outperform other deep learning-based algorithms in both short-term and long-term traffic prediction.

Neuromorphic photonic networks using silicon photonic weight banks.

PubMed

Tait, Alexander N; de Lima, Thomas Ferreira; Zhou, Ellen; Wu, Allie X; Nahmias, Mitchell A; Shastri, Bhavin J; Prucnal, Paul R

2017-08-07

Photonic systems for high-performance information processing have attracted renewed interest. Neuromorphic silicon photonics has the potential to integrate processing functions that vastly exceed the capabilities of electronics. We report first observations of a recurrent silicon photonic neural network, in which connections are configured by microring weight banks. A mathematical isomorphism between the silicon photonic circuit and a continuous neural network model is demonstrated through dynamical bifurcation analysis. Exploiting this isomorphism, a simulated 24-node silicon photonic neural network is programmed using "neural compiler" to solve a differential system emulation task. A 294-fold acceleration against a conventional benchmark is predicted. We also propose and derive power consumption analysis for modulator-class neurons that, as opposed to laser-class neurons, are compatible with silicon photonic platforms. At increased scale, Neuromorphic silicon photonics could access new regimes of ultrafast information processing for radio, control, and scientific computing.

Neuronal network model of interictal and recurrent ictal activity

NASA Astrophysics Data System (ADS)

Lopes, M. A.; Lee, K.-E.; Goltsev, A. V.

2017-12-01

We propose a neuronal network model which undergoes a saddle node on an invariant circle bifurcation as the mechanism of the transition from the interictal to the ictal (seizure) state. In the vicinity of this transition, the model captures important dynamical features of both interictal and ictal states. We study the nature of interictal spikes and early warnings of the transition predicted by this model. We further demonstrate that recurrent seizures emerge due to the interaction between two networks.

Pathological effect of homeostatic synaptic scaling on network dynamics in diseases of the cortex.

PubMed

Fröhlich, Flavio; Bazhenov, Maxim; Sejnowski, Terrence J

2008-02-13

Slow periodic EEG discharges are common in CNS disorders. The pathophysiology of this aberrant rhythmic activity is poorly understood. We used a computational model of a neocortical network with a dynamic homeostatic scaling rule to show that loss of input (partial deafferentation) can trigger network reorganization that results in pathological periodic discharges. The decrease in average firing rate in the network by deafferentation was compensated by homeostatic synaptic scaling of recurrent excitation among pyramidal cells. Synaptic scaling succeeded in recovering the network target firing rate for all degrees of deafferentation (fraction of deafferented cells), but there was a critical degree of deafferentation for pathological network reorganization. For deafferentation degrees below this value, homeostatic upregulation of recurrent excitation had minimal effect on the macroscopic network dynamics. For deafferentation above this threshold, however, a slow periodic oscillation appeared, patterns of activity were less sparse, and bursting occurred in individual neurons. Also, comparison of spike-triggered afferent and recurrent excitatory conductances revealed that information transmission was strongly impaired. These results suggest that homeostatic plasticity can lead to secondary functional impairment in case of cortical disorders associated with cell loss.

Finite-time synchronization control of a class of memristor-based recurrent neural networks.

PubMed

Jiang, Minghui; Wang, Shuangtao; Mei, Jun; Shen, Yanjun

2015-03-01

This paper presents a global and local finite-time synchronization control law for memristor neural networks. By utilizing the drive-response concept, differential inclusions theory, and Lyapunov functional method, we establish several sufficient conditions for finite-time synchronization between the master and corresponding slave memristor-based neural network with the designed controller. In comparison with the existing results, the proposed stability conditions are new, and the obtained results extend some previous works on conventional recurrent neural networks. Two numerical examples are provided to illustrate the effective of the design method. Copyright © 2014 Elsevier Ltd. All rights reserved.

A Recurrent Probabilistic Neural Network with Dimensionality Reduction Based on Time-series Discriminant Component Analysis.

PubMed

Hayashi, Hideaki; Shibanoki, Taro; Shima, Keisuke; Kurita, Yuichi; Tsuji, Toshio

2015-12-01

This paper proposes a probabilistic neural network (NN) developed on the basis of time-series discriminant component analysis (TSDCA) that can be used to classify high-dimensional time-series patterns. TSDCA involves the compression of high-dimensional time series into a lower dimensional space using a set of orthogonal transformations and the calculation of posterior probabilities based on a continuous-density hidden Markov model with a Gaussian mixture model expressed in the reduced-dimensional space. The analysis can be incorporated into an NN, which is named a time-series discriminant component network (TSDCN), so that parameters of dimensionality reduction and classification can be obtained simultaneously as network coefficients according to a backpropagation through time-based learning algorithm with the Lagrange multiplier method. The TSDCN is considered to enable high-accuracy classification of high-dimensional time-series patterns and to reduce the computation time taken for network training. The validity of the TSDCN is demonstrated for high-dimensional artificial data and electroencephalogram signals in the experiments conducted during the study.

Medical Concept Normalization in Social Media Posts with Recurrent Neural Networks.

PubMed

Tutubalina, Elena; Miftahutdinov, Zulfat; Nikolenko, Sergey; Malykh, Valentin

2018-06-12

Text mining of scientific libraries and social media has already proven itself as a reliable tool for drug repurposing and hypothesis generation. The task of mapping a disease mention to a concept in a controlled vocabulary, typically to the standard thesaurus in the Unified Medical Language System (UMLS), is known as medical concept normalization. This task is challenging due to the differences in the use of medical terminology between health care professionals and social media texts coming from the lay public. To bridge this gap, we use sequence learning with recurrent neural networks and semantic representation of one- or multi-word expressions: we develop end-to-end architectures directly tailored to the task, including bidirectional Long Short-Term Memory, Gated Recurrent Units with an attention mechanism, and additional semantic similarity features based on UMLS. Our evaluation against a standard benchmark shows that recurrent neural networks improve results over an effective baseline for classification based on convolutional neural networks. A qualitative examination of mentions discovered in a dataset of user reviews collected from popular online health information platforms as well as a quantitative evaluation both show improvements in the semantic representation of health-related expressions in social media. Copyright © 2018. Published by Elsevier Inc.

Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition.

PubMed

Ordóñez, Francisco Javier; Roggen, Daniel

2016-01-18

Human activity recognition (HAR) tasks have traditionally been solved using engineered features obtained by heuristic processes. Current research suggests that deep convolutional neural networks are suited to automate feature extraction from raw sensor inputs. However, human activities are made of complex sequences of motor movements, and capturing this temporal dynamics is fundamental for successful HAR. Based on the recent success of recurrent neural networks for time series domains, we propose a generic deep framework for activity recognition based on convolutional and LSTM recurrent units, which: (i) is suitable for multimodal wearable sensors; (ii) can perform sensor fusion naturally; (iii) does not require expert knowledge in designing features; and (iv) explicitly models the temporal dynamics of feature activations. We evaluate our framework on two datasets, one of which has been used in a public activity recognition challenge. Our results show that our framework outperforms competing deep non-recurrent networks on the challenge dataset by 4% on average; outperforming some of the previous reported results by up to 9%. Our results show that the framework can be applied to homogeneous sensor modalities, but can also fuse multimodal sensors to improve performance. We characterise key architectural hyperparameters' influence on performance to provide insights about their optimisation.

Integrative gene network construction to analyze cancer recurrence using semi-supervised learning.

PubMed

Park, Chihyun; Ahn, Jaegyoon; Kim, Hyunjin; Park, Sanghyun

2014-01-01

The prognosis of cancer recurrence is an important research area in bioinformatics and is challenging due to the small sample sizes compared to the vast number of genes. There have been several attempts to predict cancer recurrence. Most studies employed a supervised approach, which uses only a few labeled samples. Semi-supervised learning can be a great alternative to solve this problem. There have been few attempts based on manifold assumptions to reveal the detailed roles of identified cancer genes in recurrence. In order to predict cancer recurrence, we proposed a novel semi-supervised learning algorithm based on a graph regularization approach. We transformed the gene expression data into a graph structure for semi-supervised learning and integrated protein interaction data with the gene expression data to select functionally-related gene pairs. Then, we predicted the recurrence of cancer by applying a regularization approach to the constructed graph containing both labeled and unlabeled nodes. The average improvement rate of accuracy for three different cancer datasets was 24.9% compared to existing supervised and semi-supervised methods. We performed functional enrichment on the gene networks used for learning. We identified that those gene networks are significantly associated with cancer-recurrence-related biological functions. Our algorithm was developed with standard C++ and is available in Linux and MS Windows formats in the STL library. The executable program is freely available at: http://embio.yonsei.ac.kr/~Park/ssl.php.

Understanding Resilience Dimensions and Adaptive Strategies to the Impact of Recurrent Droughts in Borana Zone, Oromia Region, Ethiopia: A Grounded Theory Approach

PubMed Central

Birhanu, Zewdie; Ambelu, Argaw; Berhanu, Negalign; Tesfaye, Abraraw; Woldemichael, Kifle

2017-01-01

Recurrent shocks and stresses are increasingly deteriorating pastoralist communities’ resilience capacities in many aspects. A context specific resilience framework is essential to strengthen pastoralist community’s resilience capacity towards the impact of recurrent drought. Hence, the present study was aimed to develop a context specific and data driven resilience building framework towards impacts of recurrent droughts in the case of Borana pastoralists in Ethiopia. Qualitative grounded theory approach was employed to guide the study process. The data were collected through focus group discussions and in-depth interviews in two drought affected districts of Borana Zone during October 2013. The analysis was assisted by ATLAS. ti 7.1.4. The analysis provided a context specific resilience building conceptual tool, which consists of, closely interconnected, eight dimensions operating at multiple capacities and levels: environment (underlying vulnerability factor); livestock, infrastructures/social services, and wealth (immediate causes and effects); community network/social capital, as well as governance, peace and security (support and enabling factors oriented), psychosocial, and human capital (as eventual outcomes and impacts). The resilience capacities of these pastoralist communities have been eroded, leaving them without sufficient and effective adaptive strategies. The emergent resilience framework can serve as a useful guidance to design context-specific interventions that makes the people and the system resilient to the impacts of recurrent droughts. PMID:28134771

Understanding Resilience Dimensions and Adaptive Strategies to the Impact of Recurrent Droughts in Borana Zone, Oromia Region, Ethiopia: A Grounded Theory Approach.

PubMed

Birhanu, Zewdie; Ambelu, Argaw; Berhanu, Negalign; Tesfaye, Abraraw; Woldemichael, Kifle

2017-01-26

Recurrent shocks and stresses are increasingly deteriorating pastoralist communities' resilience capacities in many aspects. A context specific resilience framework is essential to strengthen pastoralist community's resilience capacity towards the impact of recurrent drought. Hence, the present study was aimed to develop a context specific and data driven resilience building framework towards impacts of recurrent droughts in the case of Borana pastoralists in Ethiopia. Qualitative grounded theory approach was employed to guide the study process. The data were collected through focus group discussions and in-depth interviews in two drought affected districts of Borana Zone during October 2013. The analysis was assisted by ATLAS. ti 7.1.4. The analysis provided a context specific resilience building conceptual tool, which consists of, closely interconnected, eight dimensions operating at multiple capacities and levels: environment (underlying vulnerability factor); livestock, infrastructures/social services, and wealth (immediate causes and effects); community network/social capital, as well as governance, peace and security (support and enabling factors oriented), psychosocial, and human capital (as eventual outcomes and impacts). The resilience capacities of these pastoralist communities have been eroded, leaving them without sufficient and effective adaptive strategies. The emergent resilience framework can serve as a useful guidance to design context-specific interventions that makes the people and the system resilient to the impacts of recurrent droughts.

Control of magnetic bearing systems via the Chebyshev polynomial-based unified model (CPBUM) neural network.

PubMed

Jeng, J T; Lee, T T

2000-01-01

A Chebyshev polynomial-based unified model (CPBUM) neural network is introduced and applied to control a magnetic bearing systems. First, we show that the CPBUM neural network not only has the same capability of universal approximator, but also has faster learning speed than conventional feedforward/recurrent neural network. It turns out that the CPBUM neural network is more suitable in the design of controller than the conventional feedforward/recurrent neural network. Second, we propose the inverse system method, based on the CPBUM neural networks, to control a magnetic bearing system. The proposed controller has two structures; namely, off-line and on-line learning structures. We derive a new learning algorithm for each proposed structure. The experimental results show that the proposed neural network architecture provides a greater flexibility and better performance in controlling magnetic bearing systems.

MATLAB Simulation of Gradient-Based Neural Network for Online Matrix Inversion

NASA Astrophysics Data System (ADS)

Zhang, Yunong; Chen, Ke; Ma, Weimu; Li, Xiao-Dong

This paper investigates the simulation of a gradient-based recurrent neural network for online solution of the matrix-inverse problem. Several important techniques are employed as follows to simulate such a neural system. 1) Kronecker product of matrices is introduced to transform a matrix-differential-equation (MDE) to a vector-differential-equation (VDE); i.e., finally, a standard ordinary-differential-equation (ODE) is obtained. 2) MATLAB routine "ode45" is introduced to solve the transformed initial-value ODE problem. 3) In addition to various implementation errors, different kinds of activation functions are simulated to show the characteristics of such a neural network. Simulation results substantiate the theoretical analysis and efficacy of the gradient-based neural network for online constant matrix inversion.

Delay selection by spike-timing-dependent plasticity in recurrent networks of spiking neurons receiving oscillatory inputs.

PubMed

Kerr, Robert R; Burkitt, Anthony N; Thomas, Doreen A; Gilson, Matthieu; Grayden, David B

2013-01-01

Learning rules, such as spike-timing-dependent plasticity (STDP), change the structure of networks of neurons based on the firing activity. A network level understanding of these mechanisms can help infer how the brain learns patterns and processes information. Previous studies have shown that STDP selectively potentiates feed-forward connections that have specific axonal delays, and that this underlies behavioral functions such as sound localization in the auditory brainstem of the barn owl. In this study, we investigate how STDP leads to the selective potentiation of recurrent connections with different axonal and dendritic delays during oscillatory activity. We develop analytical models of learning with additive STDP in recurrent networks driven by oscillatory inputs, and support the results using simulations with leaky integrate-and-fire neurons. Our results show selective potentiation of connections with specific axonal delays, which depended on the input frequency. In addition, we demonstrate how this can lead to a network becoming selective in the amplitude of its oscillatory response to this frequency. We extend this model of axonal delay selection within a single recurrent network in two ways. First, we show the selective potentiation of connections with a range of both axonal and dendritic delays. Second, we show axonal delay selection between multiple groups receiving out-of-phase, oscillatory inputs. We discuss the application of these models to the formation and activation of neuronal ensembles or cell assemblies in the cortex, and also to missing fundamental pitch perception in the auditory brainstem.

Delay Selection by Spike-Timing-Dependent Plasticity in Recurrent Networks of Spiking Neurons Receiving Oscillatory Inputs

PubMed Central

Kerr, Robert R.; Burkitt, Anthony N.; Thomas, Doreen A.; Gilson, Matthieu; Grayden, David B.

2013-01-01

Learning rules, such as spike-timing-dependent plasticity (STDP), change the structure of networks of neurons based on the firing activity. A network level understanding of these mechanisms can help infer how the brain learns patterns and processes information. Previous studies have shown that STDP selectively potentiates feed-forward connections that have specific axonal delays, and that this underlies behavioral functions such as sound localization in the auditory brainstem of the barn owl. In this study, we investigate how STDP leads to the selective potentiation of recurrent connections with different axonal and dendritic delays during oscillatory activity. We develop analytical models of learning with additive STDP in recurrent networks driven by oscillatory inputs, and support the results using simulations with leaky integrate-and-fire neurons. Our results show selective potentiation of connections with specific axonal delays, which depended on the input frequency. In addition, we demonstrate how this can lead to a network becoming selective in the amplitude of its oscillatory response to this frequency. We extend this model of axonal delay selection within a single recurrent network in two ways. First, we show the selective potentiation of connections with a range of both axonal and dendritic delays. Second, we show axonal delay selection between multiple groups receiving out-of-phase, oscillatory inputs. We discuss the application of these models to the formation and activation of neuronal ensembles or cell assemblies in the cortex, and also to missing fundamental pitch perception in the auditory brainstem. PMID:23408878

Neuronal network models of epileptogenesis

PubMed Central

Abdullahi, Aminu T.; Adamu, Lawan H.

2017-01-01

Epilepsy is a chronic neurological condition, following some trigger, transforming a normal brain to one that produces recurrent unprovoked seizures. In the search for the mechanisms that best explain the epileptogenic process, there is a growing body of evidence suggesting that the epilepsies are network level disorders. In this review, we briefly describe the concept of neuronal networks and highlight 2 methods used to analyse such networks. The first method, graph theory, is used to describe general characteristics of a network to facilitate comparison between normal and abnormal networks. The second, dynamic causal modelling, is useful in the analysis of the pathways of seizure spread. We concluded that the end results of the epileptogenic process are best understood as abnormalities of neuronal circuitry and not simply as molecular or cellular abnormalities. The network approach promises to generate new understanding and more targeted treatment of epilepsy. PMID:28416779

A recurrent neural-network-based sensor and actuator fault detection and isolation for nonlinear systems with application to the satellite's attitude control subsystem.

PubMed

Talebi, H A; Khorasani, K; Tafazoli, S

2009-01-01

This paper presents a robust fault detection and isolation (FDI) scheme for a general class of nonlinear systems using a neural-network-based observer strategy. Both actuator and sensor faults are considered. The nonlinear system considered is subject to both state and sensor uncertainties and disturbances. Two recurrent neural networks are employed to identify general unknown actuator and sensor faults, respectively. The neural network weights are updated according to a modified backpropagation scheme. Unlike many previous methods developed in the literature, our proposed FDI scheme does not rely on availability of full state measurements. The stability of the overall FDI scheme in presence of unknown sensor and actuator faults as well as plant and sensor noise and uncertainties is shown by using the Lyapunov's direct method. The stability analysis developed requires no restrictive assumptions on the system and/or the FDI algorithm. Magnetorquer-type actuators and magnetometer-type sensors that are commonly employed in the attitude control subsystem (ACS) of low-Earth orbit (LEO) satellites for attitude determination and control are considered in our case studies. The effectiveness and capabilities of our proposed fault diagnosis strategy are demonstrated and validated through extensive simulation studies.

Representing Where along with What Information in a Model of a Cortical Patch

PubMed Central

Roudi, Yasser; Treves, Alessandro

2008-01-01

Behaving in the real world requires flexibly combining and maintaining information about both continuous and discrete variables. In the visual domain, several lines of evidence show that neurons in some cortical networks can simultaneously represent information about the position and identity of objects, and maintain this combined representation when the object is no longer present. The underlying network mechanism for this combined representation is, however, unknown. In this paper, we approach this issue through a theoretical analysis of recurrent networks. We present a model of a cortical network that can retrieve information about the identity of objects from incomplete transient cues, while simultaneously representing their spatial position. Our results show that two factors are important in making this possible: A) a metric organisation of the recurrent connections, and B) a spatially localised change in the linear gain of neurons. Metric connectivity enables a localised retrieval of information about object identity, while gain modulation ensures localisation in the correct position. Importantly, we find that the amount of information that the network can retrieve and retain about identity is strongly affected by the amount of information it maintains about position. This balance can be controlled by global signals that change the neuronal gain. These results show that anatomical and physiological properties, which have long been known to characterise cortical networks, naturally endow them with the ability to maintain a conjunctive representation of the identity and location of objects. PMID:18369416

Ablation as targeted perturbation to rewire communication network of persistent atrial fibrillation

PubMed Central

Tao, Susumu; Way, Samuel F.; Garland, Joshua; Chrispin, Jonathan; Ciuffo, Luisa A.; Balouch, Muhammad A.; Nazarian, Saman; Spragg, David D.; Marine, Joseph E.; Berger, Ronald D.; Calkins, Hugh

2017-01-01

Persistent atrial fibrillation (AF) can be viewed as disintegrated patterns of information transmission by action potential across the communication network consisting of nodes linked by functional connectivity. To test the hypothesis that ablation of persistent AF is associated with improvement in both local and global connectivity within the communication networks, we analyzed multi-electrode basket catheter electrograms of 22 consecutive patients (63.5 ± 9.7 years, 78% male) during persistent AF before and after the focal impulse and rotor modulation-guided ablation. Eight patients (36%) developed recurrence within 6 months after ablation. We defined communication networks of AF by nodes (cardiac tissue adjacent to each electrode) and edges (mutual information between pairs of nodes). To evaluate patient-specific parameters of communication, thresholds of mutual information were applied to preserve 10% to 30% of the strongest edges. There was no significant difference in network parameters between both atria at baseline. Ablation effectively rewired the communication network of persistent AF to improve the overall connectivity. In addition, successful ablation improved local connectivity by increasing the average clustering coefficient, and also improved global connectivity by decreasing the characteristic path length. As a result, successful ablation improved the efficiency and robustness of the communication network by increasing the small-world index. These changes were not observed in patients with AF recurrence. Furthermore, a significant increase in the small-world index after ablation was associated with synchronization of the rhythm by acute AF termination. In conclusion, successful ablation rewires communication networks during persistent AF, making it more robust, efficient, and easier to synchronize. Quantitative analysis of communication networks provides not only a mechanistic insight that AF may be sustained by spatially localized sources and global connectivity, but also patient-specific metrics that could serve as a valid endpoint for therapeutic interventions. PMID:28678805

Iterative free-energy optimization for recurrent neural networks (INFERNO).

PubMed

Pitti, Alexandre; Gaussier, Philippe; Quoy, Mathias

2017-01-01

The intra-parietal lobe coupled with the Basal Ganglia forms a working memory that demonstrates strong planning capabilities for generating robust yet flexible neuronal sequences. Neurocomputational models however, often fails to control long range neural synchrony in recurrent spiking networks due to spontaneous activity. As a novel framework based on the free-energy principle, we propose to see the problem of spikes' synchrony as an optimization problem of the neurons sub-threshold activity for the generation of long neuronal chains. Using a stochastic gradient descent, a reinforcement signal (presumably dopaminergic) evaluates the quality of one input vector to move the recurrent neural network to a desired activity; depending on the error made, this input vector is strengthened to hill-climb the gradient or elicited to search for another solution. This vector can be learned then by one associative memory as a model of the basal-ganglia to control the recurrent neural network. Experiments on habit learning and on sequence retrieving demonstrate the capabilities of the dual system to generate very long and precise spatio-temporal sequences, above two hundred iterations. Its features are applied then to the sequential planning of arm movements. In line with neurobiological theories, we discuss its relevance for modeling the cortico-basal working memory to initiate flexible goal-directed neuronal chains of causation and its relation to novel architectures such as Deep Networks, Neural Turing Machines and the Free-Energy Principle.

Firing rate dynamics in recurrent spiking neural networks with intrinsic and network heterogeneity.

PubMed

Ly, Cheng

2015-12-01

Heterogeneity of neural attributes has recently gained a lot of attention and is increasing recognized as a crucial feature in neural processing. Despite its importance, this physiological feature has traditionally been neglected in theoretical studies of cortical neural networks. Thus, there is still a lot unknown about the consequences of cellular and circuit heterogeneity in spiking neural networks. In particular, combining network or synaptic heterogeneity and intrinsic heterogeneity has yet to be considered systematically despite the fact that both are known to exist and likely have significant roles in neural network dynamics. In a canonical recurrent spiking neural network model, we study how these two forms of heterogeneity lead to different distributions of excitatory firing rates. To analytically characterize how these types of heterogeneities affect the network, we employ a dimension reduction method that relies on a combination of Monte Carlo simulations and probability density function equations. We find that the relationship between intrinsic and network heterogeneity has a strong effect on the overall level of heterogeneity of the firing rates. Specifically, this relationship can lead to amplification or attenuation of firing rate heterogeneity, and these effects depend on whether the recurrent network is firing asynchronously or rhythmically firing. These observations are captured with the aforementioned reduction method, and furthermore simpler analytic descriptions based on this dimension reduction method are developed. The final analytic descriptions provide compact and descriptive formulas for how the relationship between intrinsic and network heterogeneity determines the firing rate heterogeneity dynamics in various settings.

The relevance of network micro-structure for neural dynamics.

PubMed

Pernice, Volker; Deger, Moritz; Cardanobile, Stefano; Rotter, Stefan

2013-01-01

The activity of cortical neurons is determined by the input they receive from presynaptic neurons. Many previous studies have investigated how specific aspects of the statistics of the input affect the spike trains of single neurons and neurons in recurrent networks. However, typically very simple random network models are considered in such studies. Here we use a recently developed algorithm to construct networks based on a quasi-fractal probability measure which are much more variable than commonly used network models, and which therefore promise to sample the space of recurrent networks in a more exhaustive fashion than previously possible. We use the generated graphs as the underlying network topology in simulations of networks of integrate-and-fire neurons in an asynchronous and irregular state. Based on an extensive dataset of networks and neuronal simulations we assess statistical relations between features of the network structure and the spiking activity. Our results highlight the strong influence that some details of the network structure have on the activity dynamics of both single neurons and populations, even if some global network parameters are kept fixed. We observe specific and consistent relations between activity characteristics like spike-train irregularity or correlations and network properties, for example the distributions of the numbers of in- and outgoing connections or clustering. Exploiting these relations, we demonstrate that it is possible to estimate structural characteristics of the network from activity data. We also assess higher order correlations of spiking activity in the various networks considered here, and find that their occurrence strongly depends on the network structure. These results provide directions for further theoretical studies on recurrent networks, as well as new ways to interpret spike train recordings from neural circuits.

Modeling of cortical signals using echo state networks

NASA Astrophysics Data System (ADS)

Zhou, Hanying; Wang, Yongji; Huang, Jiangshuai

2009-10-01

Diverse modeling frameworks have been utilized with the ultimate goal of translating brain cortical signals into prediction of visible behavior. The inputs to these models are usually multidimensional neural recordings collected from relevant regions of a monkey's brain while the outputs are the associated behavior which is typically the 2-D or 3-D hand position of a primate. Here our task is to set up a proper model in order to figure out the move trajectories by input the neural signals which are simultaneously collected in the experiment. In this paper, we propose to use Echo State Networks (ESN) to map the neural firing activities into hand positions. ESN is a newly developed recurrent neural network(RNN) model. Besides its dynamic property and short term memory just as other recurrent neural networks have, it has a special echo state property which endows it with the ability to model nonlinear dynamic systems powerfully. What distinguished it from transitional recurrent neural networks most significantly is its special learning method. In this paper we train this net with a refined version of its typical training method and get a better model.

Frame prediction using recurrent convolutional encoder with residual learning

NASA Astrophysics Data System (ADS)

Yue, Boxuan; Liang, Jun

2018-05-01

The prediction for the frame of a video is difficult but in urgent need in auto-driving. Conventional methods can only predict some abstract trends of the region of interest. The boom of deep learning makes the prediction for frames possible. In this paper, we propose a novel recurrent convolutional encoder and DE convolutional decoder structure to predict frames. We introduce the residual learning in the convolution encoder structure to solve the gradient issues. The residual learning can transform the gradient back propagation to an identity mapping. It can reserve the whole gradient information and overcome the gradient issues in Recurrent Neural Networks (RNN) and Convolutional Neural Networks (CNN). Besides, compared with the branches in CNNs and the gated structures in RNNs, the residual learning can save the training time significantly. In the experiments, we use UCF101 dataset to train our networks, the predictions are compared with some state-of-the-art methods. The results show that our networks can predict frames fast and efficiently. Furthermore, our networks are used for the driving video to verify the practicability.

A New Local Bipolar Autoassociative Memory Based on External Inputs of Discrete Recurrent Neural Networks With Time Delay.

PubMed

Zhou, Caigen; Zeng, Xiaoqin; Luo, Chaomin; Zhang, Huaguang

In this paper, local bipolar auto-associative memories are presented based on discrete recurrent neural networks with a class of gain type activation function. The weight parameters of neural networks are acquired by a set of inequalities without the learning procedure. The global exponential stability criteria are established to ensure the accuracy of the restored patterns by considering time delays and external inputs. The proposed methodology is capable of effectively overcoming spurious memory patterns and achieving memory capacity. The effectiveness, robustness, and fault-tolerant capability are validated by simulated experiments.In this paper, local bipolar auto-associative memories are presented based on discrete recurrent neural networks with a class of gain type activation function. The weight parameters of neural networks are acquired by a set of inequalities without the learning procedure. The global exponential stability criteria are established to ensure the accuracy of the restored patterns by considering time delays and external inputs. The proposed methodology is capable of effectively overcoming spurious memory patterns and achieving memory capacity. The effectiveness, robustness, and fault-tolerant capability are validated by simulated experiments.

LFNet: A Novel Bidirectional Recurrent Convolutional Neural Network for Light-Field Image Super-Resolution.

PubMed

Wang, Yunlong; Liu, Fei; Zhang, Kunbo; Hou, Guangqi; Sun, Zhenan; Tan, Tieniu

2018-09-01

The low spatial resolution of light-field image poses significant difficulties in exploiting its advantage. To mitigate the dependency of accurate depth or disparity information as priors for light-field image super-resolution, we propose an implicitly multi-scale fusion scheme to accumulate contextual information from multiple scales for super-resolution reconstruction. The implicitly multi-scale fusion scheme is then incorporated into bidirectional recurrent convolutional neural network, which aims to iteratively model spatial relations between horizontally or vertically adjacent sub-aperture images of light-field data. Within the network, the recurrent convolutions are modified to be more effective and flexible in modeling the spatial correlations between neighboring views. A horizontal sub-network and a vertical sub-network of the same network structure are ensembled for final outputs via stacked generalization. Experimental results on synthetic and real-world data sets demonstrate that the proposed method outperforms other state-of-the-art methods by a large margin in peak signal-to-noise ratio and gray-scale structural similarity indexes, which also achieves superior quality for human visual systems. Furthermore, the proposed method can enhance the performance of light field applications such as depth estimation.

Reverse engineering of TLX oncogenic transcriptional networks identifies RUNX1 as tumor suppressor in T-ALL.

PubMed

Della Gatta, Giusy; Palomero, Teresa; Perez-Garcia, Arianne; Ambesi-Impiombato, Alberto; Bansal, Mukesh; Carpenter, Zachary W; De Keersmaecker, Kim; Sole, Xavier; Xu, Luyao; Paietta, Elisabeth; Racevskis, Janis; Wiernik, Peter H; Rowe, Jacob M; Meijerink, Jules P; Califano, Andrea; Ferrando, Adolfo A

2012-02-26

The TLX1 and TLX3 transcription factor oncogenes have a key role in the pathogenesis of T cell acute lymphoblastic leukemia (T-ALL). Here we used reverse engineering of global transcriptional networks to decipher the oncogenic regulatory circuit controlled by TLX1 and TLX3. This systems biology analysis defined T cell leukemia homeobox 1 (TLX1) and TLX3 as master regulators of an oncogenic transcriptional circuit governing T-ALL. Notably, a network structure analysis of this hierarchical network identified RUNX1 as a key mediator of the T-ALL induced by TLX1 and TLX3 and predicted a tumor-suppressor role for RUNX1 in T cell transformation. Consistent with these results, we identified recurrent somatic loss-of-function mutations in RUNX1 in human T-ALL. Overall, these results place TLX1 and TLX3 at the top of an oncogenic transcriptional network controlling leukemia development, show the power of network analyses to identify key elements in the regulatory circuits governing human cancer and identify RUNX1 as a tumor-suppressor gene in T-ALL.

An Investigation of the Application of Artificial Neural Networks to Adaptive Optics Imaging Systems

DTIC Science & Technology

1991-12-01

neural network and the feedforward neural network studied is the single layer perceptron artificial neural network . The recurrent artificial neural network input...features are the wavefront sensor slope outputs and neighboring actuator feedback commands. The feedforward artificial neural network input

Feedforward Inhibition Allows Input Summation to Vary in Recurrent Cortical Networks

PubMed Central

2018-01-01

Abstract Brain computations depend on how neurons transform inputs to spike outputs. Here, to understand input-output transformations in cortical networks, we recorded spiking responses from visual cortex (V1) of awake mice of either sex while pairing sensory stimuli with optogenetic perturbation of excitatory and parvalbumin-positive inhibitory neurons. We found that V1 neurons’ average responses were primarily additive (linear). We used a recurrent cortical network model to determine whether these data, as well as past observations of nonlinearity, could be described by a common circuit architecture. Simulations showed that cortical input-output transformations can be changed from linear to sublinear with moderate (∼20%) strengthening of connections between inhibitory neurons, but this change away from linear scaling depends on the presence of feedforward inhibition. Simulating a variety of recurrent connection strengths showed that, compared with when input arrives only to excitatory neurons, networks produce a wider range of output spiking responses in the presence of feedforward inhibition. PMID:29682603

A network analysis of DSM-5 posttraumatic stress disorder and functional impairment in UK treatment-seeking veterans.

PubMed

Ross, Jana; Murphy, Dominic; Armour, Cherie

2018-05-28

Network analysis is a relatively new methodology for studying psychological disorders. It focuses on the associations between individual symptoms which are hypothesized to mutually interact with each other. The current study represents the first network analysis conducted with treatment-seeking military veterans in UK. The study aimed to examine the network structure of posttraumatic stress disorder (PTSD) symptoms and four domains of functional impairment by identifying the most central (i.e., important) symptoms of PTSD and by identifying those symptoms of PTSD that are related to functional impairment. Participants were 331 military veterans with probable PTSD. In the first step, a network of PTSD symptoms based on the PTSD Checklist for DSM-5 was estimated. In the second step, functional impairment items were added to the network. The most central symptoms of PTSD were recurrent thoughts, nightmares, negative emotional state, detachment and exaggerated startle response. Functional impairment was related to a number of different PTSD symptoms. Impairments in close relationships were associated primarily with the negative alterations in cognitions and mood symptoms and impairments in home management were associated primarily with the reexperiencing symptoms. The results are discussed in relation to previous PTSD network studies and include implications for clinical practice. Copyright © 2018 Elsevier Ltd. All rights reserved.

Long-term Recurrent Convolutional Networks for Visual Recognition and Description

DTIC Science & Technology

2014-11-17

deep???, are effective for tasks involving sequences, visual and otherwise. We develop a novel recurrent convolutional architecture suitable for large...models which are also recurrent, or “temporally deep”, are effective for tasks involving sequences, visual and otherwise. We develop a novel recurrent...limitation of simple RNN models which strictly integrate state information over time is known as the “vanishing gradient” effect : the ability to

A novel prosodic-information synthesizer based on recurrent fuzzy neural network for the Chinese TTS system.

PubMed

Lin, Chin-Teng; Wu, Rui-Cheng; Chang, Jyh-Yeong; Liang, Sheng-Fu

2004-02-01

In this paper, a new technique for the Chinese text-to-speech (TTS) system is proposed. Our major effort focuses on the prosodic information generation. New methodologies for constructing fuzzy rules in a prosodic model simulating human's pronouncing rules are developed. The proposed Recurrent Fuzzy Neural Network (RFNN) is a multilayer recurrent neural network (RNN) which integrates a Self-cOnstructing Neural Fuzzy Inference Network (SONFIN) into a recurrent connectionist structure. The RFNN can be functionally divided into two parts. The first part adopts the SONFIN as a prosodic model to explore the relationship between high-level linguistic features and prosodic information based on fuzzy inference rules. As compared to conventional neural networks, the SONFIN can always construct itself with an economic network size in high learning speed. The second part employs a five-layer network to generate all prosodic parameters by directly using the prosodic fuzzy rules inferred from the first part as well as other important features of syllables. The TTS system combined with the proposed method can behave not only sandhi rules but also the other prosodic phenomena existing in the traditional TTS systems. Moreover, the proposed scheme can even find out some new rules about prosodic phrase structure. The performance of the proposed RFNN-based prosodic model is verified by imbedding it into a Chinese TTS system with a Chinese monosyllable database based on the time-domain pitch synchronous overlap add (TD-PSOLA) method. Our experimental results show that the proposed RFNN can generate proper prosodic parameters including pitch means, pitch shapes, maximum energy levels, syllable duration, and pause duration. Some synthetic sounds are online available for demonstration.

On-line training of recurrent neural networks with continuous topology adaptation.

PubMed

Obradovic, D

1996-01-01

This paper presents an online procedure for training dynamic neural networks with input-output recurrences whose topology is continuously adjusted to the complexity of the target system dynamics. This is accomplished by changing the number of the elements of the network hidden layer whenever the existing topology cannot capture the dynamics presented by the new data. The training mechanism is based on the suitably altered extended Kalman filter (EKF) algorithm which is simultaneously used for the network parameter adjustment and for its state estimation. The network consists of a single hidden layer with Gaussian radial basis functions (GRBF), and a linear output layer. The choice of the GRBF is induced by the requirements of the online learning. The latter implies the network architecture which permits only local influence of the new data point in order not to forget the previously learned dynamics. The continuous topology adaptation is implemented in our algorithm to avoid memory and computational problems of using a regular grid of GRBF'S which covers the network input space. Furthermore, we show that the resulting parameter increase can be handled "smoothly" without interfering with the already acquired information. If the target system dynamics are changing over time, we show that a suitable forgetting factor can be used to "unlearn" the no longer-relevant dynamics. The quality of the recurrent network training algorithm is demonstrated on the identification of nonlinear dynamic systems.

Active Control of Complex Systems via Dynamic (Recurrent) Neural Networks

DTIC Science & Technology

1992-05-30

course, to on-going changes brought about by learning processes. As research in neurodynamics proceeded, the concept of reverberatory information flows...Microstructure of Cognition . Vol. 1: Foundations, M.I.T. Press, Cambridge, Massachusetts, pp. 354-361, 1986. 100 I Schwarz, G., "Estimating the dimension of a...Continually Running Fully Recurrent Neural Networks, ICS Report 8805, Institute of Cognitive Science, University of California at San Diego, 1988. 10 II

Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition

PubMed Central

Ordóñez, Francisco Javier; Roggen, Daniel

2016-01-01

Human activity recognition (HAR) tasks have traditionally been solved using engineered features obtained by heuristic processes. Current research suggests that deep convolutional neural networks are suited to automate feature extraction from raw sensor inputs. However, human activities are made of complex sequences of motor movements, and capturing this temporal dynamics is fundamental for successful HAR. Based on the recent success of recurrent neural networks for time series domains, we propose a generic deep framework for activity recognition based on convolutional and LSTM recurrent units, which: (i) is suitable for multimodal wearable sensors; (ii) can perform sensor fusion naturally; (iii) does not require expert knowledge in designing features; and (iv) explicitly models the temporal dynamics of feature activations. We evaluate our framework on two datasets, one of which has been used in a public activity recognition challenge. Our results show that our framework outperforms competing deep non-recurrent networks on the challenge dataset by 4% on average; outperforming some of the previous reported results by up to 9%. Our results show that the framework can be applied to homogeneous sensor modalities, but can also fuse multimodal sensors to improve performance. We characterise key architectural hyperparameters’ influence on performance to provide insights about their optimisation. PMID:26797612

Nonlinearly Activated Neural Network for Solving Time-Varying Complex Sylvester Equation.

PubMed

Li, Shuai; Li, Yangming

2013-10-28

The Sylvester equation is often encountered in mathematics and control theory. For the general time-invariant Sylvester equation problem, which is defined in the domain of complex numbers, the Bartels-Stewart algorithm and its extensions are effective and widely used with an O(n³) time complexity. When applied to solving the time-varying Sylvester equation, the computation burden increases intensively with the decrease of sampling period and cannot satisfy continuous realtime calculation requirements. For the special case of the general Sylvester equation problem defined in the domain of real numbers, gradient-based recurrent neural networks are able to solve the time-varying Sylvester equation in real time, but there always exists an estimation error while a recently proposed recurrent neural network by Zhang et al [this type of neural network is called Zhang neural network (ZNN)] converges to the solution ideally. The advancements in complex-valued neural networks cast light to extend the existing real-valued ZNN for solving the time-varying real-valued Sylvester equation to its counterpart in the domain of complex numbers. In this paper, a complex-valued ZNN for solving the complex-valued Sylvester equation problem is investigated and the global convergence of the neural network is proven with the proposed nonlinear complex-valued activation functions. Moreover, a special type of activation function with a core function, called sign-bi-power function, is proven to enable the ZNN to converge in finite time, which further enhances its advantage in online processing. In this case, the upper bound of the convergence time is also derived analytically. Simulations are performed to evaluate and compare the performance of the neural network with different parameters and activation functions. Both theoretical analysis and numerical simulations validate the effectiveness of the proposed method.

Low-complexity nonlinear adaptive filter based on a pipelined bilinear recurrent neural network.

PubMed

Zhao, Haiquan; Zeng, Xiangping; He, Zhengyou

2011-09-01

To reduce the computational complexity of the bilinear recurrent neural network (BLRNN), a novel low-complexity nonlinear adaptive filter with a pipelined bilinear recurrent neural network (PBLRNN) is presented in this paper. The PBLRNN, inheriting the modular architectures of the pipelined RNN proposed by Haykin and Li, comprises a number of BLRNN modules that are cascaded in a chained form. Each module is implemented by a small-scale BLRNN with internal dynamics. Since those modules of the PBLRNN can be performed simultaneously in a pipelined parallelism fashion, it would result in a significant improvement of computational efficiency. Moreover, due to nesting module, the performance of the PBLRNN can be further improved. To suit for the modular architectures, a modified adaptive amplitude real-time recurrent learning algorithm is derived on the gradient descent approach. Extensive simulations are carried out to evaluate the performance of the PBLRNN on nonlinear system identification, nonlinear channel equalization, and chaotic time series prediction. Experimental results show that the PBLRNN provides considerably better performance compared to the single BLRNN and RNN models.

Functional Resistance to Recurrent Spatially Heterogeneous Disturbances Is Facilitated by Increased Activity of Surviving Bacteria in a Virtual Ecosystem

PubMed Central

König, Sara; Worrich, Anja; Banitz, Thomas; Harms, Hauke; Kästner, Matthias; Miltner, Anja; Wick, Lukas Y.; Frank, Karin; Thullner, Martin; Centler, Florian

2018-01-01

Bacterial degradation of organic compounds is an important ecosystem function with relevance to, e.g., the cycling of elements or the degradation of organic contaminants. It remains an open question, however, to which extent ecosystems are able to maintain such biodegradation function under recurrent disturbances (functional resistance) and how this is related to the bacterial biomass abundance. In this paper, we use a numerical simulation approach to systematically analyze the dynamic response of a microbial population to recurrent disturbances of different spatial distribution. The spatially explicit model considers microbial degradation, growth, dispersal, and spatial networks that facilitate bacterial dispersal mimicking effects of mycelial networks in nature. We find: (i) There is a certain capacity for high resistance of biodegradation performance to recurrent disturbances. (ii) If this resistance capacity is exceeded, spatial zones of different biodegradation performance develop, ranging from no or reduced to even increased performance. (iii) Bacterial biomass and biodegradation dynamics respond inversely to the spatial fragmentation of disturbances: overall biodegradation performance improves with increasing fragmentation, but bacterial biomass declines. (iv) Bacterial dispersal networks can enhance functional resistance against recurrent disturbances, mainly by reactivating zones in the core of disturbed areas, even though this leads to an overall reduction of bacterial biomass. PMID:29696013

Current treatment of ocular toxoplasmosis in immunocompetent patients: A network meta-analysis.

PubMed

Zhang, Yanxia; Lin, Xiao; Lu, Fangli

2018-04-25

Ocular toxoplasmosis (OT) is the most frequent form of infectious posterior uveitis caused by the protozoan parasite Toxoplasma gondii. To evaluate the available evidence in peer-reviewed publications about the most effective therapy for OT in immunocompetent patients, herein a systematic literature search was conducted using Embase, PubMed, Google Scholar, and the Cochrane Central Register of Controlled Trials (CENTRAL) database from January 1987 to October 2017, with search terms "OT", "retinochoroiditis", "treatment", and "immunocompetent"; search filters "controlled clinical trial", "randomized clinical trial", and "clinical trial". The included studies were performed to evaluate the various treatment modalities of OT. Different treatment regimens were compared with regard to the improvement of visual acuity, the resolution of vitreous inflammation, recurrence, and side-effects. We independently extracted data and assessed eligibility and risk of bias using the preferred reporting items for systematic reviews and meta-analysis, and resolved any disagreement through discussion. A Bayesian network meta-analysis model was used to evaluate the interesting outcomes of all the interventions. Total 10 trials of treatments for OT were found to meet the inclusion criteria. Six trials of treatments including clindamycin, azithromycin, and trimethoprim-sulfamethoxazole (TMP-SMX) were compared with conventional therapy (the combination of pyrimethamine, sulfadiazine, and prednisone) for evaluation of the effect on visual acuity, vitreous inflammation, recurrence of OT, and side-effects. Two trials were compared TMP-SMX with placebo. One trial was compared azithromycin with TMP-SMX. And another trial was compared among treatments with clindamycin, P-S, TMP-SMX, and placebo. Based on our network meta-analysis, therapy with TMP-SMX seems to be an alternative treatment of OT in immunocompetent patients. Copyright © 2018 Elsevier B.V. All rights reserved.

Consistency Analysis of Genome-Scale Models of Bacterial Metabolism: A Metamodel Approach

PubMed Central

Ponce-de-Leon, Miguel; Calle-Espinosa, Jorge; Peretó, Juli; Montero, Francisco

2015-01-01

Genome-scale metabolic models usually contain inconsistencies that manifest as blocked reactions and gap metabolites. With the purpose to detect recurrent inconsistencies in metabolic models, a large-scale analysis was performed using a previously published dataset of 130 genome-scale models. The results showed that a large number of reactions (~22%) are blocked in all the models where they are present. To unravel the nature of such inconsistencies a metamodel was construed by joining the 130 models in a single network. This metamodel was manually curated using the unconnected modules approach, and then, it was used as a reference network to perform a gap-filling on each individual genome-scale model. Finally, a set of 36 models that had not been considered during the construction of the metamodel was used, as a proof of concept, to extend the metamodel with new biochemical information, and to assess its impact on gap-filling results. The analysis performed on the metamodel allowed to conclude: 1) the recurrent inconsistencies found in the models were already present in the metabolic database used during the reconstructions process; 2) the presence of inconsistencies in a metabolic database can be propagated to the reconstructed models; 3) there are reactions not manifested as blocked which are active as a consequence of some classes of artifacts, and; 4) the results of an automatic gap-filling are highly dependent on the consistency and completeness of the metamodel or metabolic database used as the reference network. In conclusion the consistency analysis should be applied to metabolic databases in order to detect and fill gaps as well as to detect and remove artifacts and redundant information. PMID:26629901

A comparison of effectiveness among frequent treatments of recurrent spontaneous abortion: A Bayesian network meta-analysis.

PubMed

Lv, Sha; Yu, Jing; Xu, Xiaoxiao

2018-04-30

A comprehensive network meta-analysis was designed to clarify contradictions and offer valuable clinical guidance in the treatment of recurrent spontaneous abortion (RSA). The included clinical trials were selected from the relevant medical journal databases and screened. Treatments were ranked by the surface under the cumulative ranking curve. Heat plots were constructed to analyze the inconsistency between direct data and network results, and adjusted funnel plots were constructed to assess publication bias. Forty-nine randomized controlled trials involving a total of 8496 RSA patients were selected. With placebo as control, corticosteroid plus low dose aspirin (LDA) plus unfractionated heparin (UFH), granulocyte colony-stimulating factor (G-CSF) alone, and LDA plus low molecular weight heparin (LMWH) all demonstrated effectiveness in increasing successful live birth rates and reducing the incidences of miscarriage. However, no treatment was demonstrably superior to placebo in terms of pregnancy success. For all 3 endpoints (live birth, abortion and success pregnancy), the adjusted funnel plots were symmetric to zero and indicated no publication bias. In terms of live birth and abortion rates, no treatment outperformed placebo in patients with antiphospholipid syndrome. In consideration of live birth and abortion rates, corticosteroid plus LDA plus UFH appeared to be the optimum treatment strategy; G-CSF was second, followed by LDA with LMWH, LDA plus LMWH plus intravenous immunoglobulin, corticosteroid with LDA and others. Subgroup analysis demonstrated no benefit of antithrombotic therapy in patients with antiphospholipid syndrome. © 2018 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.

A one-layer recurrent neural network for constrained pseudoconvex optimization and its application for dynamic portfolio optimization.

PubMed

Liu, Qingshan; Guo, Zhishan; Wang, Jun

2012-02-01

In this paper, a one-layer recurrent neural network is proposed for solving pseudoconvex optimization problems subject to linear equality and bound constraints. Compared with the existing neural networks for optimization (e.g., the projection neural networks), the proposed neural network is capable of solving more general pseudoconvex optimization problems with equality and bound constraints. Moreover, it is capable of solving constrained fractional programming problems as a special case. The convergence of the state variables of the proposed neural network to achieve solution optimality is guaranteed as long as the designed parameters in the model are larger than the derived lower bounds. Numerical examples with simulation results illustrate the effectiveness and characteristics of the proposed neural network. In addition, an application for dynamic portfolio optimization is discussed. Copyright © 2011 Elsevier Ltd. All rights reserved.

Spike-train spectra and network response functions for non-linear integrate-and-fire neurons.

PubMed

Richardson, Magnus J E

2008-11-01

Reduced models have long been used as a tool for the analysis of the complex activity taking place in neurons and their coupled networks. Recent advances in experimental and theoretical techniques have further demonstrated the usefulness of this approach. Despite the often gross simplification of the underlying biophysical properties, reduced models can still present significant difficulties in their analysis, with the majority of exact and perturbative results available only for the leaky integrate-and-fire model. Here an elementary numerical scheme is demonstrated which can be used to calculate a number of biologically important properties of the general class of non-linear integrate-and-fire models. Exact results for the first-passage-time density and spike-train spectrum are derived, as well as the linear response properties and emergent states of recurrent networks. Given that the exponential integrate-fire model has recently been shown to agree closely with the experimentally measured response of pyramidal cells, the methodology presented here promises to provide a convenient tool to facilitate the analysis of cortical-network dynamics.

Forecasting the discomfort levels within the greater Athens area, Greece using artificial neural networks and multiple criteria analysis

NASA Astrophysics Data System (ADS)

Vouterakos, P. A.; Moustris, K. P.; Bartzokas, A.; Ziomas, I. C.; Nastos, P. T.; Paliatsos, A. G.

2012-12-01

In this work, artificial neural networks (ANNs) were developed and applied in order to forecast the discomfort levels due to the combination of high temperature and air humidity, during the hot season of the year, in eight different regions within the Greater Athens area (GAA), Greece. For the selection of the best type and architecture of ANNs-forecasting models, the multiple criteria analysis (MCA) technique was applied. Three different types of ANNs were developed and tested with the MCA method. Concretely, the multilayer perceptron, the generalized feed forward networks (GFFN), and the time-lag recurrent networks were developed and tested. Results showed that the best ANNs type performance was achieved by using the GFFN model for the prediction of discomfort levels due to high temperature and air humidity within GAA. For the evaluation of the constructed ANNs, appropriate statistical indices were used. The analysis proved that the forecasting ability of the developed ANNs models is very satisfactory at a significant statistical level of p < 0.01.

A neural-network-based model for the dynamic simulation of the tire/suspension system while traversing road irregularities.

PubMed

Guarneri, Paolo; Rocca, Gianpiero; Gobbi, Massimiliano

2008-09-01

This paper deals with the simulation of the tire/suspension dynamics by using recurrent neural networks (RNNs). RNNs are derived from the multilayer feedforward neural networks, by adding feedback connections between output and input layers. The optimal network architecture derives from a parametric analysis based on the optimal tradeoff between network accuracy and size. The neural network can be trained with experimental data obtained in the laboratory from simulated road profiles (cleats). The results obtained from the neural network demonstrate good agreement with the experimental results over a wide range of operation conditions. The NN model can be effectively applied as a part of vehicle system model to accurately predict elastic bushings and tire dynamics behavior. Although the neural network model, as a black-box model, does not provide a good insight of the physical behavior of the tire/suspension system, it is a useful tool for assessing vehicle ride and noise, vibration, harshness (NVH) performance due to its good computational efficiency and accuracy.

Application of recurrence quantification analysis for the automated identification of epileptic EEG signals.

PubMed

Acharya, U Rajendra; Sree, S Vinitha; Chattopadhyay, Subhagata; Yu, Wenwei; Ang, Peng Chuan Alvin

2011-06-01

Epilepsy is a common neurological disorder that is characterized by the recurrence of seizures. Electroencephalogram (EEG) signals are widely used to diagnose seizures. Because of the non-linear and dynamic nature of the EEG signals, it is difficult to effectively decipher the subtle changes in these signals by visual inspection and by using linear techniques. Therefore, non-linear methods are being researched to analyze the EEG signals. In this work, we use the recorded EEG signals in Recurrence Plots (RP), and extract Recurrence Quantification Analysis (RQA) parameters from the RP in order to classify the EEG signals into normal, ictal, and interictal classes. Recurrence Plot (RP) is a graph that shows all the times at which a state of the dynamical system recurs. Studies have reported significantly different RQA parameters for the three classes. However, more studies are needed to develop classifiers that use these promising features and present good classification accuracy in differentiating the three types of EEG segments. Therefore, in this work, we have used ten RQA parameters to quantify the important features in the EEG signals.These features were fed to seven different classifiers: Support vector machine (SVM), Gaussian Mixture Model (GMM), Fuzzy Sugeno Classifier, K-Nearest Neighbor (KNN), Naive Bayes Classifier (NBC), Decision Tree (DT), and Radial Basis Probabilistic Neural Network (RBPNN). Our results show that the SVM classifier was able to identify the EEG class with an average efficiency of 95.6%, sensitivity and specificity of 98.9% and 97.8%, respectively.

Computing by robust transience: How the fronto-parietal network performs sequential category-based decisions

PubMed Central

Chaisangmongkon, Warasinee; Swaminathan, Sruthi K.; Freedman, David J.; Wang, Xiao-Jing

2017-01-01

Summary Decision making involves dynamic interplay between internal judgements and external perception, which has been investigated in delayed match-to-category (DMC) experiments. Our analysis of neural recordings shows that, during DMC tasks, LIP and PFC neurons demonstrate mixed, time-varying, and heterogeneous selectivity, but previous theoretical work has not established the link between these neural characteristics and population-level computations. We trained a recurrent network model to perform DMC tasks and found that the model can remarkably reproduce key features of neuronal selectivity at the single-neuron and population levels. Analysis of the trained networks elucidates that robust transient trajectories of the neural population are the key driver of sequential categorical decisions. The directions of trajectories are governed by network self-organized connectivity, defining a ‘neural landscape’, consisting of a task-tailored arrangement of slow states and dynamical tunnels. With this model, we can identify functionally-relevant circuit motifs and generalize the framework to solve other categorization tasks. PMID:28334612

Locking of correlated neural activity to ongoing oscillations

PubMed Central

Helias, Moritz

2017-01-01

Population-wide oscillations are ubiquitously observed in mesoscopic signals of cortical activity. In these network states a global oscillatory cycle modulates the propensity of neurons to fire. Synchronous activation of neurons has been hypothesized to be a separate channel of signal processing information in the brain. A salient question is therefore if and how oscillations interact with spike synchrony and in how far these channels can be considered separate. Experiments indeed showed that correlated spiking co-modulates with the static firing rate and is also tightly locked to the phase of beta-oscillations. While the dependence of correlations on the mean rate is well understood in feed-forward networks, it remains unclear why and by which mechanisms correlations tightly lock to an oscillatory cycle. We here demonstrate that such correlated activation of pairs of neurons is qualitatively explained by periodically-driven random networks. We identify the mechanisms by which covariances depend on a driving periodic stimulus. Mean-field theory combined with linear response theory yields closed-form expressions for the cyclostationary mean activities and pairwise zero-time-lag covariances of binary recurrent random networks. Two distinct mechanisms cause time-dependent covariances: the modulation of the susceptibility of single neurons (via the external input and network feedback) and the time-varying variances of single unit activities. For some parameters, the effectively inhibitory recurrent feedback leads to resonant covariances even if mean activities show non-resonant behavior. Our analytical results open the question of time-modulated synchronous activity to a quantitative analysis. PMID:28604771

Learning Orthographic Structure With Sequential Generative Neural Networks.

PubMed

Testolin, Alberto; Stoianov, Ivilin; Sperduti, Alessandro; Zorzi, Marco

2016-04-01

Learning the structure of event sequences is a ubiquitous problem in cognition and particularly in language. One possible solution is to learn a probabilistic generative model of sequences that allows making predictions about upcoming events. Though appealing from a neurobiological standpoint, this approach is typically not pursued in connectionist modeling. Here, we investigated a sequential version of the restricted Boltzmann machine (RBM), a stochastic recurrent neural network that extracts high-order structure from sensory data through unsupervised generative learning and can encode contextual information in the form of internal, distributed representations. We assessed whether this type of network can extract the orthographic structure of English monosyllables by learning a generative model of the letter sequences forming a word training corpus. We show that the network learned an accurate probabilistic model of English graphotactics, which can be used to make predictions about the letter following a given context as well as to autonomously generate high-quality pseudowords. The model was compared to an extended version of simple recurrent networks, augmented with a stochastic process that allows autonomous generation of sequences, and to non-connectionist probabilistic models (n-grams and hidden Markov models). We conclude that sequential RBMs and stochastic simple recurrent networks are promising candidates for modeling cognition in the temporal domain. Copyright © 2015 Cognitive Science Society, Inc.

Inferring causal genomic alterations in breast cancer using gene expression data

PubMed Central

2011-01-01

Background One of the primary objectives in cancer research is to identify causal genomic alterations, such as somatic copy number variation (CNV) and somatic mutations, during tumor development. Many valuable studies lack genomic data to detect CNV; therefore, methods that are able to infer CNVs from gene expression data would help maximize the value of these studies. Results We developed a framework for identifying recurrent regions of CNV and distinguishing the cancer driver genes from the passenger genes in the regions. By inferring CNV regions across many datasets we were able to identify 109 recurrent amplified/deleted CNV regions. Many of these regions are enriched for genes involved in many important processes associated with tumorigenesis and cancer progression. Genes in these recurrent CNV regions were then examined in the context of gene regulatory networks to prioritize putative cancer driver genes. The cancer driver genes uncovered by the framework include not only well-known oncogenes but also a number of novel cancer susceptibility genes validated via siRNA experiments. Conclusions To our knowledge, this is the first effort to systematically identify and validate drivers for expression based CNV regions in breast cancer. The framework where the wavelet analysis of copy number alteration based on expression coupled with the gene regulatory network analysis, provides a blueprint for leveraging genomic data to identify key regulatory components and gene targets. This integrative approach can be applied to many other large-scale gene expression studies and other novel types of cancer data such as next-generation sequencing based expression (RNA-Seq) as well as CNV data. PMID:21806811

Modeling and control of magnetorheological fluid dampers using neural networks

NASA Astrophysics Data System (ADS)

Wang, D. H.; Liao, W. H.

2005-02-01

Due to the inherent nonlinear nature of magnetorheological (MR) fluid dampers, one of the challenging aspects for utilizing these devices to achieve high system performance is the development of accurate models and control algorithms that can take advantage of their unique characteristics. In this paper, the direct identification and inverse dynamic modeling for MR fluid dampers using feedforward and recurrent neural networks are studied. The trained direct identification neural network model can be used to predict the damping force of the MR fluid damper on line, on the basis of the dynamic responses across the MR fluid damper and the command voltage, and the inverse dynamic neural network model can be used to generate the command voltage according to the desired damping force through supervised learning. The architectures and the learning methods of the dynamic neural network models and inverse neural network models for MR fluid dampers are presented, and some simulation results are discussed. Finally, the trained neural network models are applied to predict and control the damping force of the MR fluid damper. Moreover, validation methods for the neural network models developed are proposed and used to evaluate their performance. Validation results with different data sets indicate that the proposed direct identification dynamic model using the recurrent neural network can be used to predict the damping force accurately and the inverse identification dynamic model using the recurrent neural network can act as a damper controller to generate the command voltage when the MR fluid damper is used in a semi-active mode.

A Radial Basis Function Approach to Financial Time Series Analysis

DTIC Science & Technology

1993-12-01

including efficient methods for parameter estimation and pruning, a pointwise prediction error estimator, and a methodology for controlling the "data...collection of practical techniques to address these issues for a modeling methodology . Radial Basis Function networks. These techniques in- clude efficient... methodology often then amounts to a careful consideration of the interplay between model complexity and reliability. These will be recurrent themes

A signal-flow-graph approach to on-line gradient calculation.

PubMed

Campolucci, P; Uncini, A; Piazza, F

2000-08-01

A large class of nonlinear dynamic adaptive systems such as dynamic recurrent neural networks can be effectively represented by signal flow graphs (SFGs). By this method, complex systems are described as a general connection of many simple components, each of them implementing a simple one-input, one-output transformation, as in an electrical circuit. Even if graph representations are popular in the neural network community, they are often used for qualitative description rather than for rigorous representation and computational purposes. In this article, a method for both on-line and batch-backward gradient computation of a system output or cost function with respect to system parameters is derived by the SFG representation theory and its known properties. The system can be any causal, in general nonlinear and time-variant, dynamic system represented by an SFG, in particular any feedforward, time-delay, or recurrent neural network. In this work, we use discrete-time notation, but the same theory holds for the continuous-time case. The gradient is obtained in a straightforward way by the analysis of two SFGs, the original one and its adjoint (obtained from the first by simple transformations), without the complex chain rule expansions of derivatives usually employed. This method can be used for sensitivity analysis and for learning both off-line and on-line. On-line learning is particularly important since it is required by many real applications, such as digital signal processing, system identification and control, channel equalization, and predistortion.

Artificial intelligence estimates the impact of human papillomavirus types in influencing the risk of cervical dysplasia recurrence: progress toward a more personalized approach.

PubMed

Bogani, Giorgio; Ditto, Antonino; Martinelli, Fabio; Signorelli, Mauro; Chiappa, Valentina; Leone Roberti Maggiore, Umberto; Taverna, Francesca; Lombardo, Claudia; Borghi, Chiara; Scaffa, Cono; Lorusso, Domenica; Raspagliesi, Francesco

2018-01-22

The objective of this study was to determine whether the pretreatment human papillomavirus (HPV) genotype might predict the risk of cervical dysplasia persistence/recurrence. Retrospective analysis of prospectively collected data of consecutive 5104 women who underwent the HPV-DNA test were matched with retrospective data of women undergoing either follow-up or medical/surgical treatment(s) for genital HPV-related infection(s). Artificial neuronal network (ANN) analysis was used in order to weight the importance of different HPV genotypes in predicting cervical dysplasia persistence/recurrence. ANN simulates a biological neuronal system from both the structural and functional points of view: like neurons, ANN acquires knowledge through a learning-phase process and allows weighting the importance of covariates, thus establishing how much a variable influences a multifactor phenomenon. Overall, 5104 women were tested for HPV. Among them, 1273 (25%) patients underwent treatment for HPV-related disorders. LASER conization and cervical vaporization were performed in 807 (59%) and 386 (30%) patients, respectively, and secondary cervical conization in 45 (5.5%). ANN technology showed that the most important genotypes predicting cervical dysplasia persistence/recurrence were HPV-16 (normalized importance: 100%), HPV-59 (normalized importance: 51.2%), HPV-52 (normalized importance: 47.7%), HPV-18 (normalized importance: 32.8%) and HPV-45 (normalized importance: 30.2%). The pretreatment diagnosis of all of those genotypes, except HPV-45, correlated with an increased risk of cervical dysplasia persistence/recurrence; the pretreatment diagnosis was also arrived at using standard univariate and multivariable models (P<0.01). Pretreatment positivity for HPV-16, HPV-18, HPV-52 and HPV-59 might correlate with an increased risk of cervical dysplasia persistence/recurrence after treatment. These data might be helpful during patients' counseling and to implement new vaccination programs.

Video Super-Resolution via Bidirectional Recurrent Convolutional Networks.

PubMed

Huang, Yan; Wang, Wei; Wang, Liang

2018-04-01

Super resolving a low-resolution video, namely video super-resolution (SR), is usually handled by either single-image SR or multi-frame SR. Single-Image SR deals with each video frame independently, and ignores intrinsic temporal dependency of video frames which actually plays a very important role in video SR. Multi-Frame SR generally extracts motion information, e.g., optical flow, to model the temporal dependency, but often shows high computational cost. Considering that recurrent neural networks (RNNs) can model long-term temporal dependency of video sequences well, we propose a fully convolutional RNN named bidirectional recurrent convolutional network for efficient multi-frame SR. Different from vanilla RNNs, 1) the commonly-used full feedforward and recurrent connections are replaced with weight-sharing convolutional connections. So they can greatly reduce the large number of network parameters and well model the temporal dependency in a finer level, i.e., patch-based rather than frame-based, and 2) connections from input layers at previous timesteps to the current hidden layer are added by 3D feedforward convolutions, which aim to capture discriminate spatio-temporal patterns for short-term fast-varying motions in local adjacent frames. Due to the cheap convolutional operations, our model has a low computational complexity and runs orders of magnitude faster than other multi-frame SR methods. With the powerful temporal dependency modeling, our model can super resolve videos with complex motions and achieve well performance.

Multiplex Recurrence Networks

NASA Astrophysics Data System (ADS)

Eroglu, Deniz; Marwan, Norbert

2017-04-01

The complex nature of a variety of phenomena in physical, biological, or earth sciences is driven by a large number of degrees of freedom which are strongly interconnected. Although the evolution of such systems is described by multivariate time series (MTS), so far research mostly focuses on analyzing these components one by one. Recurrence based analyses are powerful methods to understand the underlying dynamics of a dynamical system and have been used for many successful applications including examples from earth science, economics, or chemical reactions. The backbone of these techniques is creating the phase space of the system. However, increasing the dimension of a system requires increasing the length of the time series in order get significant and reliable results. This requirement is one of the challenges in many disciplines, in particular in palaeoclimate, thus, it is not easy to create a phase space from measured MTS due to the limited number of available obervations (samples). To overcome this problem, we suggest to create recurrence networks from each component of the system and combine them into a multiplex network structure, the multiplex recurrence network (MRN). We test the MRN by using prototypical mathematical models and demonstrate its use by studying high-dimensional palaeoclimate dynamics derived from pollen data from the Bear Lake (Utah, US). By using the MRN, we can distinguish typical climate transition events, e.g., such between Marine Isotope Stages.

DEEP MOTIF DASHBOARD: VISUALIZING AND UNDERSTANDING GENOMIC SEQUENCES USING DEEP NEURAL NETWORKS.

PubMed

Lanchantin, Jack; Singh, Ritambhara; Wang, Beilun; Qi, Yanjun

2017-01-01

Deep neural network (DNN) models have recently obtained state-of-the-art prediction accuracy for the transcription factor binding (TFBS) site classification task. However, it remains unclear how these approaches identify meaningful DNA sequence signals and give insights as to why TFs bind to certain locations. In this paper, we propose a toolkit called the Deep Motif Dashboard (DeMo Dashboard) which provides a suite of visualization strategies to extract motifs, or sequence patterns from deep neural network models for TFBS classification. We demonstrate how to visualize and understand three important DNN models: convolutional, recurrent, and convolutional-recurrent networks. Our first visualization method is finding a test sequence's saliency map which uses first-order derivatives to describe the importance of each nucleotide in making the final prediction. Second, considering recurrent models make predictions in a temporal manner (from one end of a TFBS sequence to the other), we introduce temporal output scores, indicating the prediction score of a model over time for a sequential input. Lastly, a class-specific visualization strategy finds the optimal input sequence for a given TFBS positive class via stochastic gradient optimization. Our experimental results indicate that a convolutional-recurrent architecture performs the best among the three architectures. The visualization techniques indicate that CNN-RNN makes predictions by modeling both motifs as well as dependencies among them.

Deep Motif Dashboard: Visualizing and Understanding Genomic Sequences Using Deep Neural Networks

PubMed Central

Lanchantin, Jack; Singh, Ritambhara; Wang, Beilun; Qi, Yanjun

2018-01-01

Deep neural network (DNN) models have recently obtained state-of-the-art prediction accuracy for the transcription factor binding (TFBS) site classification task. However, it remains unclear how these approaches identify meaningful DNA sequence signals and give insights as to why TFs bind to certain locations. In this paper, we propose a toolkit called the Deep Motif Dashboard (DeMo Dashboard) which provides a suite of visualization strategies to extract motifs, or sequence patterns from deep neural network models for TFBS classification. We demonstrate how to visualize and understand three important DNN models: convolutional, recurrent, and convolutional-recurrent networks. Our first visualization method is finding a test sequence’s saliency map which uses first-order derivatives to describe the importance of each nucleotide in making the final prediction. Second, considering recurrent models make predictions in a temporal manner (from one end of a TFBS sequence to the other), we introduce temporal output scores, indicating the prediction score of a model over time for a sequential input. Lastly, a class-specific visualization strategy finds the optimal input sequence for a given TFBS positive class via stochastic gradient optimization. Our experimental results indicate that a convolutional-recurrent architecture performs the best among the three architectures. The visualization techniques indicate that CNN-RNN makes predictions by modeling both motifs as well as dependencies among them. PMID:27896980

Complete stability of delayed recurrent neural networks with Gaussian activation functions.

PubMed

Liu, Peng; Zeng, Zhigang; Wang, Jun

2017-01-01

This paper addresses the complete stability of delayed recurrent neural networks with Gaussian activation functions. By means of the geometrical properties of Gaussian function and algebraic properties of nonsingular M-matrix, some sufficient conditions are obtained to ensure that for an n-neuron neural network, there are exactly 3 k equilibrium points with 0≤k≤n, among which 2 k and 3 k -2 k equilibrium points are locally exponentially stable and unstable, respectively. Moreover, it concludes that all the states converge to one of the equilibrium points; i.e., the neural networks are completely stable. The derived conditions herein can be easily tested. Finally, a numerical example is given to illustrate the theoretical results. Copyright © 2016 Elsevier Ltd. All rights reserved.

Geometric and dynamic perspectives on phase-coherent and noncoherent chaos.

PubMed

Zou, Yong; Donner, Reik V; Kurths, Jürgen

2012-03-01

Statistically distinguishing between phase-coherent and noncoherent chaotic dynamics from time series is a contemporary problem in nonlinear sciences. In this work, we propose different measures based on recurrence properties of recorded trajectories, which characterize the underlying systems from both geometric and dynamic viewpoints. The potentials of the individual measures for discriminating phase-coherent and noncoherent chaotic oscillations are discussed. A detailed numerical analysis is performed for the chaotic Rössler system, which displays both types of chaos as one control parameter is varied, and the Mackey-Glass system as an example of a time-delay system with noncoherent chaos. Our results demonstrate that especially geometric measures from recurrence network analysis are well suited for tracing transitions between spiral- and screw-type chaos, a common route from phase-coherent to noncoherent chaos also found in other nonlinear oscillators. A detailed explanation of the observed behavior in terms of attractor geometry is given.

Weighted gene co-expression network analysis of gene modules for the prognosis of esophageal cancer.

PubMed

Zhang, Cong; Sun, Qian

2017-06-01

Esophageal cancer is a common malignant tumor, whose pathogenesis and prognosis factors are not fully understood. This study aimed to discover the gene clusters that have similar functions and can be used to predict the prognosis of esophageal cancer. The matched microarray and RNA sequencing data of 185 patients with esophageal cancer were downloaded from The Cancer Genome Atlas (TCGA), and gene co-expression networks were built without distinguishing between squamous carcinoma and adenocarcinoma. The result showed that 12 modules were associated with one or more survival data such as recurrence status, recurrence time, vital status or vital time. Furthermore, survival analysis showed that 5 out of the 12 modules were related to progression-free survival (PFS) or overall survival (OS). As the most important module, the midnight blue module with 82 genes was related to PFS, apart from the patient age, tumor grade, primary treatment success, and duration of smoking and tumor histological type. Gene ontology enrichment analysis revealed that "glycoprotein binding" was the top enriched function of midnight blue module genes. Additionally, the blue module was the exclusive gene clusters related to OS. Platelet activating factor receptor (PTAFR) and feline Gardner-Rasheed (FGR) were the top hub genes in both modeling datasets and the STRING protein interaction database. In conclusion, our study provides novel insights into the prognosis-associated genes and screens out candidate biomarkers for esophageal cancer.

Oscillation, Conduction Delays, and Learning Cooperate to Establish Neural Competition in Recurrent Networks

PubMed Central

Kato, Hideyuki; Ikeguchi, Tohru

2016-01-01

Specific memory might be stored in a subnetwork consisting of a small population of neurons. To select neurons involved in memory formation, neural competition might be essential. In this paper, we show that excitable neurons are competitive and organize into two assemblies in a recurrent network with spike timing-dependent synaptic plasticity (STDP) and axonal conduction delays. Neural competition is established by the cooperation of spontaneously induced neural oscillation, axonal conduction delays, and STDP. We also suggest that the competition mechanism in this paper is one of the basic functions required to organize memory-storing subnetworks into fine-scale cortical networks. PMID:26840529

A theory of cerebellar cortex and adaptive motor control based on two types of universal function approximation capability.

PubMed

Fujita, Masahiko

2016-03-01

Lesions of the cerebellum result in large errors in movements. The cerebellum adaptively controls the strength and timing of motor command signals depending on the internal and external environments of movements. The present theory describes how the cerebellar cortex can control signals for accurate and timed movements. A model network of the cerebellar Golgi and granule cells is shown to be equivalent to a multiple-input (from mossy fibers) hierarchical neural network with a single hidden layer of threshold units (granule cells) that receive a common recurrent inhibition (from a Golgi cell). The weighted sum of the hidden unit signals (Purkinje cell output) is theoretically analyzed regarding the capability of the network to perform two types of universal function approximation. The hidden units begin firing as the excitatory inputs exceed the recurrent inhibition. This simple threshold feature leads to the first approximation theory, and the network final output can be any continuous function of the multiple inputs. When the input is constant, this output becomes stationary. However, when the recurrent unit activity is triggered to decrease or the recurrent inhibition is triggered to increase through a certain mechanism (metabotropic modulation or extrasynaptic spillover), the network can generate any continuous signals for a prolonged period of change in the activity of recurrent signals, as the second approximation theory shows. By incorporating the cerebellar capability of two such types of approximations to a motor system, in which learning proceeds through repeated movement trials with accompanying corrections, accurate and timed responses for reaching the target can be adaptively acquired. Simple models of motor control can solve the motor error vs. sensory error problem, as well as the structural aspects of credit (or error) assignment problem. Two physiological experiments are proposed for examining the delay and trace conditioning of eyelid responses, as well as saccade adaptation, to investigate this novel idea of cerebellar processing. Copyright © 2015 Elsevier Ltd. All rights reserved.

Real-time parallel processing of grammatical structure in the fronto-striatal system: a recurrent network simulation study using reservoir computing.

PubMed

Hinaut, Xavier; Dominey, Peter Ford

2013-01-01

Sentence processing takes place in real-time. Previous words in the sentence can influence the processing of the current word in the timescale of hundreds of milliseconds. Recent neurophysiological studies in humans suggest that the fronto-striatal system (frontal cortex, and striatum--the major input locus of the basal ganglia) plays a crucial role in this process. The current research provides a possible explanation of how certain aspects of this real-time processing can occur, based on the dynamics of recurrent cortical networks, and plasticity in the cortico-striatal system. We simulate prefrontal area BA47 as a recurrent network that receives on-line input about word categories during sentence processing, with plastic connections between cortex and striatum. We exploit the homology between the cortico-striatal system and reservoir computing, where recurrent frontal cortical networks are the reservoir, and plastic cortico-striatal synapses are the readout. The system is trained on sentence-meaning pairs, where meaning is coded as activation in the striatum corresponding to the roles that different nouns and verbs play in the sentences. The model learns an extended set of grammatical constructions, and demonstrates the ability to generalize to novel constructions. It demonstrates how early in the sentence, a parallel set of predictions are made concerning the meaning, which are then confirmed or updated as the processing of the input sentence proceeds. It demonstrates how on-line responses to words are influenced by previous words in the sentence, and by previous sentences in the discourse, providing new insight into the neurophysiology of the P600 ERP scalp response to grammatical complexity. This demonstrates that a recurrent neural network can decode grammatical structure from sentences in real-time in order to generate a predictive representation of the meaning of the sentences. This can provide insight into the underlying mechanisms of human cortico-striatal function in sentence processing.

Real-Time Parallel Processing of Grammatical Structure in the Fronto-Striatal System: A Recurrent Network Simulation Study Using Reservoir Computing

PubMed Central

Hinaut, Xavier; Dominey, Peter Ford

2013-01-01

Sentence processing takes place in real-time. Previous words in the sentence can influence the processing of the current word in the timescale of hundreds of milliseconds. Recent neurophysiological studies in humans suggest that the fronto-striatal system (frontal cortex, and striatum – the major input locus of the basal ganglia) plays a crucial role in this process. The current research provides a possible explanation of how certain aspects of this real-time processing can occur, based on the dynamics of recurrent cortical networks, and plasticity in the cortico-striatal system. We simulate prefrontal area BA47 as a recurrent network that receives on-line input about word categories during sentence processing, with plastic connections between cortex and striatum. We exploit the homology between the cortico-striatal system and reservoir computing, where recurrent frontal cortical networks are the reservoir, and plastic cortico-striatal synapses are the readout. The system is trained on sentence-meaning pairs, where meaning is coded as activation in the striatum corresponding to the roles that different nouns and verbs play in the sentences. The model learns an extended set of grammatical constructions, and demonstrates the ability to generalize to novel constructions. It demonstrates how early in the sentence, a parallel set of predictions are made concerning the meaning, which are then confirmed or updated as the processing of the input sentence proceeds. It demonstrates how on-line responses to words are influenced by previous words in the sentence, and by previous sentences in the discourse, providing new insight into the neurophysiology of the P600 ERP scalp response to grammatical complexity. This demonstrates that a recurrent neural network can decode grammatical structure from sentences in real-time in order to generate a predictive representation of the meaning of the sentences. This can provide insight into the underlying mechanisms of human cortico-striatal function in sentence processing. PMID:23383296

Generalised Transfer Functions of Neural Networks

NASA Astrophysics Data System (ADS)

Fung, C. F.; Billings, S. A.; Zhang, H.

1997-11-01

When artificial neural networks are used to model non-linear dynamical systems, the system structure which can be extremely useful for analysis and design, is buried within the network architecture. In this paper, explicit expressions for the frequency response or generalised transfer functions of both feedforward and recurrent neural networks are derived in terms of the network weights. The derivation of the algorithm is established on the basis of the Taylor series expansion of the activation functions used in a particular neural network. This leads to a representation which is equivalent to the non-linear recursive polynomial model and enables the derivation of the transfer functions to be based on the harmonic expansion method. By mapping the neural network into the frequency domain information about the structure of the underlying non-linear system can be recovered. Numerical examples are included to demonstrate the application of the new algorithm. These examples show that the frequency response functions appear to be highly sensitive to the network topology and training, and that the time domain properties fail to reveal deficiencies in the trained network structure.

Regression equations for estimating flood flows for the 2-, 10-, 25-, 50-, 100-, and 500-Year recurrence intervals in Connecticut

USGS Publications Warehouse

Ahearn, Elizabeth A.

2004-01-01

Multiple linear-regression equations were developed to estimate the magnitudes of floods in Connecticut for recurrence intervals ranging from 2 to 500 years. The equations can be used for nonurban, unregulated stream sites in Connecticut with drainage areas ranging from about 2 to 715 square miles. Flood-frequency data and hydrologic characteristics from 70 streamflow-gaging stations and the upstream drainage basins were used to develop the equations. The hydrologic characteristics?drainage area, mean basin elevation, and 24-hour rainfall?are used in the equations to estimate the magnitude of floods. Average standard errors of prediction for the equations are 31.8, 32.7, 34.4, 35.9, 37.6 and 45.0 percent for the 2-, 10-, 25-, 50-, 100-, and 500-year recurrence intervals, respectively. Simplified equations using only one hydrologic characteristic?drainage area?also were developed. The regression analysis is based on generalized least-squares regression techniques. Observed flows (log-Pearson Type III analysis of the annual maximum flows) from five streamflow-gaging stations in urban basins in Connecticut were compared to flows estimated from national three-parameter and seven-parameter urban regression equations. The comparison shows that the three- and seven- parameter equations used in conjunction with the new statewide equations generally provide reasonable estimates of flood flows for urban sites in Connecticut, although a national urban flood-frequency study indicated that the three-parameter equations significantly underestimated flood flows in many regions of the country. Verification of the accuracy of the three-parameter or seven-parameter national regression equations using new data from Connecticut stations was beyond the scope of this study. A technique for calculating flood flows at streamflow-gaging stations using a weighted average also is described. Two estimates of flood flows?one estimate based on the log-Pearson Type III analyses of the annual maximum flows at the gaging station, and the other estimate from the regression equation?are weighted together based on the years of record at the gaging station and the equivalent years of record value determined from the regression. Weighted averages of flood flows for the 2-, 10-, 25-, 50-, 100-, and 500-year recurrence intervals are tabulated for the 70 streamflow-gaging stations used in the regression analysis. Generally, weighted averages give the most accurate estimate of flood flows at gaging stations. An evaluation of the Connecticut's streamflow-gaging network was performed to determine whether the spatial coverage and range of geographic and hydrologic conditions are adequately represented for transferring flood characteristics from gaged to ungaged sites. Fifty-one of 54 stations in the current (2004) network support one or more flood needs of federal, state, and local agencies. Twenty-five of 54 stations in the current network are considered high-priority stations by the U.S. Geological Survey because of their contribution to the longterm understanding of floods, and their application for regionalflood analysis. Enhancements to the network to improve overall effectiveness for regionalization can be made by increasing the spatial coverage of gaging stations, establishing stations in regions of the state that are not well-represented, and adding stations in basins with drainage area sizes not represented. Additionally, the usefulness of the network for characterizing floods can be maintained and improved by continuing operation at the current stations because flood flows can be more accurately estimated at stations with continuous, long-term record.

Mean-field equations for neuronal networks with arbitrary degree distributions.

PubMed

Nykamp, Duane Q; Friedman, Daniel; Shaker, Sammy; Shinn, Maxwell; Vella, Michael; Compte, Albert; Roxin, Alex

2017-04-01

The emergent dynamics in networks of recurrently coupled spiking neurons depends on the interplay between single-cell dynamics and network topology. Most theoretical studies on network dynamics have assumed simple topologies, such as connections that are made randomly and independently with a fixed probability (Erdös-Rényi network) (ER) or all-to-all connected networks. However, recent findings from slice experiments suggest that the actual patterns of connectivity between cortical neurons are more structured than in the ER random network. Here we explore how introducing additional higher-order statistical structure into the connectivity can affect the dynamics in neuronal networks. Specifically, we consider networks in which the number of presynaptic and postsynaptic contacts for each neuron, the degrees, are drawn from a joint degree distribution. We derive mean-field equations for a single population of homogeneous neurons and for a network of excitatory and inhibitory neurons, where the neurons can have arbitrary degree distributions. Through analysis of the mean-field equations and simulation of networks of integrate-and-fire neurons, we show that such networks have potentially much richer dynamics than an equivalent ER network. Finally, we relate the degree distributions to so-called cortical motifs.

Mean-field equations for neuronal networks with arbitrary degree distributions

NASA Astrophysics Data System (ADS)

Nykamp, Duane Q.; Friedman, Daniel; Shaker, Sammy; Shinn, Maxwell; Vella, Michael; Compte, Albert; Roxin, Alex

2017-04-01

The emergent dynamics in networks of recurrently coupled spiking neurons depends on the interplay between single-cell dynamics and network topology. Most theoretical studies on network dynamics have assumed simple topologies, such as connections that are made randomly and independently with a fixed probability (Erdös-Rényi network) (ER) or all-to-all connected networks. However, recent findings from slice experiments suggest that the actual patterns of connectivity between cortical neurons are more structured than in the ER random network. Here we explore how introducing additional higher-order statistical structure into the connectivity can affect the dynamics in neuronal networks. Specifically, we consider networks in which the number of presynaptic and postsynaptic contacts for each neuron, the degrees, are drawn from a joint degree distribution. We derive mean-field equations for a single population of homogeneous neurons and for a network of excitatory and inhibitory neurons, where the neurons can have arbitrary degree distributions. Through analysis of the mean-field equations and simulation of networks of integrate-and-fire neurons, we show that such networks have potentially much richer dynamics than an equivalent ER network. Finally, we relate the degree distributions to so-called cortical motifs.

Finite-time convergent recurrent neural network with a hard-limiting activation function for constrained optimization with piecewise-linear objective functions.

PubMed

Liu, Qingshan; Wang, Jun

2011-04-01

This paper presents a one-layer recurrent neural network for solving a class of constrained nonsmooth optimization problems with piecewise-linear objective functions. The proposed neural network is guaranteed to be globally convergent in finite time to the optimal solutions under a mild condition on a derived lower bound of a single gain parameter in the model. The number of neurons in the neural network is the same as the number of decision variables of the optimization problem. Compared with existing neural networks for optimization, the proposed neural network has a couple of salient features such as finite-time convergence and a low model complexity. Specific models for two important special cases, namely, linear programming and nonsmooth optimization, are also presented. In addition, applications to the shortest path problem and constrained least absolute deviation problem are discussed with simulation results to demonstrate the effectiveness and characteristics of the proposed neural network.

Estimating network parameters from combined dynamics of firing rate and irregularity of single neurons.

PubMed

Hamaguchi, Kosuke; Riehle, Alexa; Brunel, Nicolas

2011-01-01

High firing irregularity is a hallmark of cortical neurons in vivo, and modeling studies suggest a balance of excitation and inhibition is necessary to explain this high irregularity. Such a balance must be generated, at least partly, from local interconnected networks of excitatory and inhibitory neurons, but the details of the local network structure are largely unknown. The dynamics of the neural activity depends on the local network structure; this in turn suggests the possibility of estimating network structure from the dynamics of the firing statistics. Here we report a new method to estimate properties of the local cortical network from the instantaneous firing rate and irregularity (CV(2)) under the assumption that recorded neurons are a part of a randomly connected sparse network. The firing irregularity, measured in monkey motor cortex, exhibits two features; many neurons show relatively stable firing irregularity in time and across different task conditions; the time-averaged CV(2) is widely distributed from quasi-regular to irregular (CV(2) = 0.3-1.0). For each recorded neuron, we estimate the three parameters of a local network [balance of local excitation-inhibition, number of recurrent connections per neuron, and excitatory postsynaptic potential (EPSP) size] that best describe the dynamics of the measured firing rates and irregularities. Our analysis shows that optimal parameter sets form a two-dimensional manifold in the three-dimensional parameter space that is confined for most of the neurons to the inhibition-dominated region. High irregularity neurons tend to be more strongly connected to the local network, either in terms of larger EPSP and inhibitory PSP size or larger number of recurrent connections, compared with the low irregularity neurons, for a given excitatory/inhibitory balance. Incorporating either synaptic short-term depression or conductance-based synapses leads many low CV(2) neurons to move to the excitation-dominated region as well as to an increase of EPSP size.

Neural-network-based state of health diagnostics for an automated radioxenon sampler/analyzer

NASA Astrophysics Data System (ADS)

Keller, Paul E.; Kangas, Lars J.; Hayes, James C.; Schrom, Brian T.; Suarez, Reynold; Hubbard, Charles W.; Heimbigner, Tom R.; McIntyre, Justin I.

2009-05-01

Artificial neural networks (ANNs) are used to determine the state-of-health (SOH) of the Automated Radioxenon Analyzer/Sampler (ARSA). ARSA is a gas collection and analysis system used for non-proliferation monitoring in detecting radioxenon released during nuclear tests. SOH diagnostics are important for automated, unmanned sensing systems so that remote detection and identification of problems can be made without onsite staff. Both recurrent and feed-forward ANNs are presented. The recurrent ANN is trained to predict sensor values based on current valve states, which control air flow, so that with only valve states the normal SOH sensor values can be predicted. Deviation between modeled value and actual is an indication of a potential problem. The feed-forward ANN acts as a nonlinear version of principal components analysis (PCA) and is trained to replicate the normal SOH sensor values. Because of ARSA's complexity, this nonlinear PCA is better able to capture the relationships among the sensors than standard linear PCA and is applicable to both sensor validation and recognizing off-normal operating conditions. Both models provide valuable information to detect impending malfunctions before they occur to avoid unscheduled shutdown. Finally, the ability of ANN methods to predict the system state is presented.

Intelligent fault diagnosis of rolling bearings using an improved deep recurrent neural network

NASA Astrophysics Data System (ADS)

Jiang, Hongkai; Li, Xingqiu; Shao, Haidong; Zhao, Ke

2018-06-01

Traditional intelligent fault diagnosis methods for rolling bearings heavily depend on manual feature extraction and feature selection. For this purpose, an intelligent deep learning method, named the improved deep recurrent neural network (DRNN), is proposed in this paper. Firstly, frequency spectrum sequences are used as inputs to reduce the input size and ensure good robustness. Secondly, DRNN is constructed by the stacks of the recurrent hidden layer to automatically extract the features from the input spectrum sequences. Thirdly, an adaptive learning rate is adopted to improve the training performance of the constructed DRNN. The proposed method is verified with experimental rolling bearing data, and the results confirm that the proposed method is more effective than traditional intelligent fault diagnosis methods.

Neural network for nonsmooth pseudoconvex optimization with general convex constraints.

PubMed

Bian, Wei; Ma, Litao; Qin, Sitian; Xue, Xiaoping

2018-05-01

In this paper, a one-layer recurrent neural network is proposed for solving a class of nonsmooth, pseudoconvex optimization problems with general convex constraints. Based on the smoothing method, we construct a new regularization function, which does not depend on any information of the feasible region. Thanks to the special structure of the regularization function, we prove the global existence, uniqueness and "slow solution" character of the state of the proposed neural network. Moreover, the state solution of the proposed network is proved to be convergent to the feasible region in finite time and to the optimal solution set of the related optimization problem subsequently. In particular, the convergence of the state to an exact optimal solution is also considered in this paper. Numerical examples with simulation results are given to show the efficiency and good characteristics of the proposed network. In addition, some preliminary theoretical analysis and application of the proposed network for a wider class of dynamic portfolio optimization are included. Copyright © 2018 Elsevier Ltd. All rights reserved.

Recurrent myocardial infarction: Mechanisms of free-floating adaptation and autonomic derangement in networked cardiac neural control.

PubMed

Kember, Guy; Ardell, Jeffrey L; Shivkumar, Kalyanam; Armour, J Andrew

2017-01-01

The cardiac nervous system continuously controls cardiac function whether or not pathology is present. While myocardial infarction typically has a major and catastrophic impact, population studies have shown that longer-term risk for recurrent myocardial infarction and the related potential for sudden cardiac death depends mainly upon standard atherosclerotic variables and autonomic nervous system maladaptations. Investigative neurocardiology has demonstrated that autonomic control of cardiac function includes local circuit neurons for networked control within the peripheral nervous system. The structural and adaptive characteristics of such networked interactions define the dynamics and a new normal for cardiac control that results in the aftermath of recurrent myocardial infarction and/or unstable angina that may or may not precipitate autonomic derangement. These features are explored here via a mathematical model of cardiac regulation. A main observation is that the control environment during pathology is an extrapolation to a setting outside prior experience. Although global bounds guarantee stability, the resulting closed-loop dynamics exhibited while the network adapts during pathology are aptly described as 'free-floating' in order to emphasize their dependence upon details of the network structure. The totality of the results provide a mechanistic reasoning that validates the clinical practice of reducing sympathetic efferent neuronal tone while aggressively targeting autonomic derangement in the treatment of ischemic heart disease.

Nonlinear dynamic systems identification using recurrent interval type-2 TSK fuzzy neural network - A novel structure.

PubMed

El-Nagar, Ahmad M

2018-01-01

In this study, a novel structure of a recurrent interval type-2 Takagi-Sugeno-Kang (TSK) fuzzy neural network (FNN) is introduced for nonlinear dynamic and time-varying systems identification. It combines the type-2 fuzzy sets (T2FSs) and a recurrent FNN to avoid the data uncertainties. The fuzzy firing strengths in the proposed structure are returned to the network input as internal variables. The interval type-2 fuzzy sets (IT2FSs) is used to describe the antecedent part for each rule while the consequent part is a TSK-type, which is a linear function of the internal variables and the external inputs with interval weights. All the type-2 fuzzy rules for the proposed RIT2TSKFNN are learned on-line based on structure and parameter learning, which are performed using the type-2 fuzzy clustering. The antecedent and consequent parameters of the proposed RIT2TSKFNN are updated based on the Lyapunov function to achieve network stability. The obtained results indicate that our proposed network has a small root mean square error (RMSE) and a small integral of square error (ISE) with a small number of rules and a small computation time compared with other type-2 FNNs. Copyright © 2017 ISA. Published by Elsevier Ltd. All rights reserved.

Functional connectivity analysis in resting state fMRI with echo-state networks and non-metric clustering for network structure recovery

NASA Astrophysics Data System (ADS)

Wismüller, Axel; DSouza, Adora M.; Abidin, Anas Z.; Wang, Xixi; Hobbs, Susan K.; Nagarajan, Mahesh B.

2015-03-01

Echo state networks (ESN) are recurrent neural networks where the hidden layer is replaced with a fixed reservoir of neurons. Unlike feed-forward networks, neuron training in ESN is restricted to the output neurons alone thereby providing a computational advantage. We demonstrate the use of such ESNs in our mutual connectivity analysis (MCA) framework for recovering the primary motor cortex network associated with hand movement from resting state functional MRI (fMRI) data. Such a framework consists of two steps - (1) defining a pair-wise affinity matrix between different pixel time series within the brain to characterize network activity and (2) recovering network components from the affinity matrix with non-metric clustering. Here, ESNs are used to evaluate pair-wise cross-estimation performance between pixel time series to create the affinity matrix, which is subsequently subject to non-metric clustering with the Louvain method. For comparison, the ground truth of the motor cortex network structure is established with a task-based fMRI sequence. Overlap between the primary motor cortex network recovered with our model free MCA approach and the ground truth was measured with the Dice coefficient. Our results show that network recovery with our proposed MCA approach is in close agreement with the ground truth. Such network recovery is achieved without requiring low-pass filtering of the time series ensembles prior to analysis, an fMRI preprocessing step that has courted controversy in recent years. Thus, we conclude our MCA framework can allow recovery and visualization of the underlying functionally connected networks in the brain on resting state fMRI.

Chaos of radiative heat-loss-induced flame front instability.

PubMed

Kinugawa, Hikaru; Ueda, Kazuhiro; Gotoda, Hiroshi

2016-03-01

We are intensively studying the chaos via the period-doubling bifurcation cascade in radiative heat-loss-induced flame front instability by analytical methods based on dynamical systems theory and complex networks. Significant changes in flame front dynamics in the chaotic region, which cannot be seen in the bifurcation diagrams, were successfully extracted from recurrence quantification analysis and nonlinear forecasting and from the network entropy. The temporal dynamics of the fuel concentration in the well-developed chaotic region is much more complicated than that of the flame front temperature. It exhibits self-affinity as a result of the scale-free structure in the constructed visibility graph.

Turkish Migrant Women with Recurrent Depression: Results from Community-based Self-help Groups.

PubMed

Siller, Heidi; Renner, Walter; Juen, Barbara

2017-01-01

The study focuses on psychosocial functioning of female Turkish immigrants in Austria with recurrent depressive disorder participating in self-help groups. Self-help groups guided by group leaders of Turkish descent should increase autonomy in participants, providing the opportunity to follow their ethnic health beliefs. Turkish immigrant women (n = 43) with recurrent depressive disorder participated in self-help groups over four months. Qualitative data of participants and group leaders, containing interviews, group protocols and supervision protocols of group leaders were analyzed using the qualitative content analysis for effects on psychosocial function, such as interaction with others, illness beliefs and benefit from self-help group. Women reported feelings of being neglected and violated by their husbands. They stated that they had gained strength and had emancipated themselves from their husbands. Self-help groups functioned as social resources and support for changes in participants' lives. Further interventions should integrate the functional value of depressive symptoms and focus on social support systems and social networks.

Towards representation of a perceptual color manifold using associative memory for color constancy.

PubMed

Seow, Ming-Jung; Asari, Vijayan K

2009-01-01

In this paper, we propose the concept of a manifold of color perception through empirical observation that the center-surround properties of images in a perceptually similar environment define a manifold in the high dimensional space. Such a manifold representation can be learned using a novel recurrent neural network based learning algorithm. Unlike the conventional recurrent neural network model in which the memory is stored in an attractive fixed point at discrete locations in the state space, the dynamics of the proposed learning algorithm represent memory as a nonlinear line of attraction. The region of convergence around the nonlinear line is defined by the statistical characteristics of the training data. This learned manifold can then be used as a basis for color correction of the images having different color perception to the learned color perception. Experimental results show that the proposed recurrent neural network learning algorithm is capable of color balance the lighting variations in images captured in different environments successfully.

Indirect adaptive fuzzy wavelet neural network with self- recurrent consequent part for AC servo system.

PubMed

Hou, Runmin; Wang, Li; Gao, Qiang; Hou, Yuanglong; Wang, Chao

2017-09-01

This paper proposes a novel indirect adaptive fuzzy wavelet neural network (IAFWNN) to control the nonlinearity, wide variations in loads, time-variation and uncertain disturbance of the ac servo system. In the proposed approach, the self-recurrent wavelet neural network (SRWNN) is employed to construct an adaptive self-recurrent consequent part for each fuzzy rule of TSK fuzzy model. For the IAFWNN controller, the online learning algorithm is based on back propagation (BP) algorithm. Moreover, an improved particle swarm optimization (IPSO) is used to adapt the learning rate. The aid of an adaptive SRWNN identifier offers the real-time gradient information to the adaptive fuzzy wavelet neural controller to overcome the impact of parameter variations, load disturbances and other uncertainties effectively, and has a good dynamic. The asymptotical stability of the system is guaranteed by using the Lyapunov method. The result of the simulation and the prototype test prove that the proposed are effective and suitable. Copyright © 2017. Published by Elsevier Ltd.

Deep RNNs for video denoising

NASA Astrophysics Data System (ADS)

Chen, Xinyuan; Song, Li; Yang, Xiaokang

2016-09-01

Video denoising can be described as the problem of mapping from a specific length of noisy frames to clean one. We propose a deep architecture based on Recurrent Neural Network (RNN) for video denoising. The model learns a patch-based end-to-end mapping between the clean and noisy video sequences. It takes the corrupted video sequences as the input and outputs the clean one. Our deep network, which we refer to as deep Recurrent Neural Networks (deep RNNs or DRNNs), stacks RNN layers where each layer receives the hidden state of the previous layer as input. Experiment shows (i) the recurrent architecture through temporal domain extracts motion information and does favor to video denoising, and (ii) deep architecture have large enough capacity for expressing mapping relation between corrupted videos as input and clean videos as output, furthermore, (iii) the model has generality to learned different mappings from videos corrupted by different types of noise (e.g., Poisson-Gaussian noise). By training on large video databases, we are able to compete with some existing video denoising methods.

YoTube: Searching Action Proposal Via Recurrent and Static Regression Networks

NASA Astrophysics Data System (ADS)

Zhu, Hongyuan; Vial, Romain; Lu, Shijian; Peng, Xi; Fu, Huazhu; Tian, Yonghong; Cao, Xianbin

2018-06-01

In this paper, we present YoTube-a novel network fusion framework for searching action proposals in untrimmed videos, where each action proposal corresponds to a spatialtemporal video tube that potentially locates one human action. Our method consists of a recurrent YoTube detector and a static YoTube detector, where the recurrent YoTube explores the regression capability of RNN for candidate bounding boxes predictions using learnt temporal dynamics and the static YoTube produces the bounding boxes using rich appearance cues in a single frame. Both networks are trained using rgb and optical flow in order to fully exploit the rich appearance, motion and temporal context, and their outputs are fused to produce accurate and robust proposal boxes. Action proposals are finally constructed by linking these boxes using dynamic programming with a novel trimming method to handle the untrimmed video effectively and efficiently. Extensive experiments on the challenging UCF-101 and UCF-Sports datasets show that our proposed technique obtains superior performance compared with the state-of-the-art.

Transcriptional Network Analysis Identifies BACH1 as a Master Regulator of Breast Cancer Bone Metastasis

PubMed Central

Liang, Yajun; Wu, Heng; Lei, Rong; Chong, Robert A.; Wei, Yong; Lu, Xin; Tagkopoulos, Ilias; Kung, Sun-Yuan; Yang, Qifeng; Hu, Guohong; Kang, Yibin

2012-01-01

The application of functional genomic analysis of breast cancer metastasis has led to the identification of a growing number of organ-specific metastasis genes, which often function in concert to facilitate different steps of the metastatic cascade. However, the gene regulatory network that controls the expression of these metastasis genes remains largely unknown. Here, we demonstrate a computational approach for the deconvolution of transcriptional networks to discover master regulators of breast cancer bone metastasis. Several known regulators of breast cancer bone metastasis such as Smad4 and HIF1 were identified in our analysis. Experimental validation of the networks revealed BACH1, a basic leucine zipper transcription factor, as the common regulator of several functional metastasis genes, including MMP1 and CXCR4. Ectopic expression of BACH1 enhanced the malignance of breast cancer cells, and conversely, BACH1 knockdown significantly reduced bone metastasis. The expression of BACH1 and its target genes was linked to the higher risk of breast cancer recurrence in patients. This study established BACH1 as the master regulator of breast cancer bone metastasis and provided a paradigm to identify molecular determinants in complex pathological processes. PMID:22875853

Attractor neural networks with resource-efficient synaptic connectivity

NASA Astrophysics Data System (ADS)

Pehlevan, Cengiz; Sengupta, Anirvan

Memories are thought to be stored in the attractor states of recurrent neural networks. Here we explore how resource constraints interplay with memory storage function to shape synaptic connectivity of attractor networks. We propose that given a set of memories, in the form of population activity patterns, the neural circuit choses a synaptic connectivity configuration that minimizes a resource usage cost. We argue that the total synaptic weight (l1-norm) in the network measures the resource cost because synaptic weight is correlated with synaptic volume, which is a limited resource, and is proportional to neurotransmitter release and post-synaptic current, both of which cost energy. Using numerical simulations and replica theory, we characterize optimal connectivity profiles in resource-efficient attractor networks. Our theory explains several experimental observations on cortical connectivity profiles, 1) connectivity is sparse, because synapses are costly, 2) bidirectional connections are overrepresented and 3) are stronger, because attractor states need strong recurrence.

Once upon a (slow) time in the land of recurrent neuronal networks….

PubMed

Huang, Chengcheng; Doiron, Brent

2017-10-01

The brain must both react quickly to new inputs as well as store a memory of past activity. This requires biology that operates over a vast range of time scales. Fast time scales are determined by the kinetics of synaptic conductances and ionic channels; however, the mechanics of slow time scales are more complicated. In this opinion article we review two distinct network-based mechanisms that impart slow time scales in recurrently coupled neuronal networks. The first is in strongly coupled networks where the time scale of the internally generated fluctuations diverges at the transition between stable and chaotic firing rate activity. The second is in networks with finitely many members where noise-induced transitions between metastable states appear as a slow time scale in the ongoing network firing activity. We discuss these mechanisms with an emphasis on their similarities and differences. Copyright © 2017 Elsevier Ltd. All rights reserved.

Stimulus-specific adaptation in a recurrent network model of primary auditory cortex

PubMed Central

2017-01-01

Stimulus-specific adaptation (SSA) occurs when neurons decrease their responses to frequently-presented (standard) stimuli but not, or not as much, to other, rare (deviant) stimuli. SSA is present in all mammalian species in which it has been tested as well as in birds. SSA confers short-term memory to neuronal responses, and may lie upstream of the generation of mismatch negativity (MMN), an important human event-related potential. Previously published models of SSA mostly rely on synaptic depression of the feedforward, thalamocortical input. Here we study SSA in a recurrent neural network model of primary auditory cortex. When the recurrent, intracortical synapses display synaptic depression, the network generates population spikes (PSs). SSA occurs in this network when deviants elicit a PS but standards do not, and we demarcate the regions in parameter space that allow SSA. While SSA based on PSs does not require feedforward depression, we identify feedforward depression as a mechanism for expanding the range of parameters that support SSA. We provide predictions for experiments that could help differentiate between SSA due to synaptic depression of feedforward connections and SSA due to synaptic depression of recurrent connections. Similar to experimental data, the magnitude of SSA in the model depends on the frequency difference between deviant and standard, probability of the deviant, inter-stimulus interval and input amplitude. In contrast to models based on feedforward depression, our model shows true deviance sensitivity as found in experiments. PMID:28288158

Short-Term Memory in Orthogonal Neural Networks

NASA Astrophysics Data System (ADS)

White, Olivia L.; Lee, Daniel D.; Sompolinsky, Haim

2004-04-01

We study the ability of linear recurrent networks obeying discrete time dynamics to store long temporal sequences that are retrievable from the instantaneous state of the network. We calculate this temporal memory capacity for both distributed shift register and random orthogonal connectivity matrices. We show that the memory capacity of these networks scales with system size.

Recurrent network dynamics reconciles visual motion segmentation and integration.

PubMed

Medathati, N V Kartheek; Rankin, James; Meso, Andrew I; Kornprobst, Pierre; Masson, Guillaume S

2017-09-12

In sensory systems, a range of computational rules are presumed to be implemented by neuronal subpopulations with different tuning functions. For instance, in primate cortical area MT, different classes of direction-selective cells have been identified and related either to motion integration, segmentation or transparency. Still, how such different tuning properties are constructed is unclear. The dominant theoretical viewpoint based on a linear-nonlinear feed-forward cascade does not account for their complex temporal dynamics and their versatility when facing different input statistics. Here, we demonstrate that a recurrent network model of visual motion processing can reconcile these different properties. Using a ring network, we show how excitatory and inhibitory interactions can implement different computational rules such as vector averaging, winner-take-all or superposition. The model also captures ordered temporal transitions between these behaviors. In particular, depending on the inhibition regime the network can switch from motion integration to segmentation, thus being able to compute either a single pattern motion or to superpose multiple inputs as in motion transparency. We thus demonstrate that recurrent architectures can adaptively give rise to different cortical computational regimes depending upon the input statistics, from sensory flow integration to segmentation.

Centralized and decentralized global outer-synchronization of asymmetric recurrent time-varying neural network by data-sampling.

PubMed

Lu, Wenlian; Zheng, Ren; Chen, Tianping

2016-03-01

In this paper, we discuss outer-synchronization of the asymmetrically connected recurrent time-varying neural networks. By using both centralized and decentralized discretization data sampling principles, we derive several sufficient conditions based on three vector norms to guarantee that the difference of any two trajectories starting from different initial values of the neural network converges to zero. The lower bounds of the common time intervals between data samples in centralized and decentralized principles are proved to be positive, which guarantees exclusion of Zeno behavior. A numerical example is provided to illustrate the efficiency of the theoretical results. Copyright © 2015 Elsevier Ltd. All rights reserved.

Postdiagnosis social networks and breast cancer mortality in the After Breast Cancer Pooling Project.

PubMed

Kroenke, Candyce H; Michael, Yvonne L; Poole, Elizabeth M; Kwan, Marilyn L; Nechuta, Sarah; Leas, Eric; Caan, Bette J; Pierce, John; Shu, Xiao-Ou; Zheng, Ying; Chen, Wendy Y

2017-04-01

Large social networks have been associated with better overall survival, though not consistently with breast cancer (BC)-specific outcomes. This study evaluated associations of postdiagnosis social networks and BC outcomes in a large cohort. Women from the After Breast Cancer Pooling Project (n = 9267) provided data on social networks within approximately 2 years of their diagnosis. A social network index was derived from information about the presence of a spouse/partner, religious ties, community ties, friendship ties, and numbers of living first-degree relatives. Cox models were used to evaluate associations, and a meta-analysis was used to determine whether effect estimates differed by cohort. Stratification by demographic, social, tumor, and treatment factors was performed. There were 1448 recurrences and 1521 deaths (990 due to BC). Associations were similar in 3 of 4 cohorts. After covariate adjustments, socially isolated women (small networks) had higher risks of recurrence (hazard ratio [HR], 1.43; 95% confidence interval [CI], 1.15-1.77), BC-specific mortality (HR, 1.64; 95% CI, 1.33-2.03), and total mortality (HR, 1.69; 95% CI, 1.43-1.99) than socially integrated women; associations were stronger in those with stage I/II cancer. In the fourth cohort, there were no significant associations with BC-specific outcomes. A lack of a spouse/partner (P = .02) and community ties (P = .04) predicted higher BC-specific mortality in older white women but not in other women. However, a lack of relatives (P = .02) and friendship ties (P = .01) predicted higher BC-specific mortality in nonwhite women only. In a large pooled cohort, larger social networks were associated with better BC-specific and overall survival. Clinicians should assess social network information as a marker of prognosis because critical supports may differ with sociodemographic factors. Cancer 2017;123:1228-1237. © 2016 American Cancer Society. © 2016 American Cancer Society.

Effect of Heterogeneity on Decorrelation Mechanisms in Spiking Neural Networks: A Neuromorphic-Hardware Study

NASA Astrophysics Data System (ADS)

Pfeil, Thomas; Jordan, Jakob; Tetzlaff, Tom; Grübl, Andreas; Schemmel, Johannes; Diesmann, Markus; Meier, Karlheinz

2016-04-01

High-level brain function, such as memory, classification, or reasoning, can be realized by means of recurrent networks of simplified model neurons. Analog neuromorphic hardware constitutes a fast and energy-efficient substrate for the implementation of such neural computing architectures in technical applications and neuroscientific research. The functional performance of neural networks is often critically dependent on the level of correlations in the neural activity. In finite networks, correlations are typically inevitable due to shared presynaptic input. Recent theoretical studies have shown that inhibitory feedback, abundant in biological neural networks, can actively suppress these shared-input correlations and thereby enable neurons to fire nearly independently. For networks of spiking neurons, the decorrelating effect of inhibitory feedback has so far been explicitly demonstrated only for homogeneous networks of neurons with linear subthreshold dynamics. Theory, however, suggests that the effect is a general phenomenon, present in any system with sufficient inhibitory feedback, irrespective of the details of the network structure or the neuronal and synaptic properties. Here, we investigate the effect of network heterogeneity on correlations in sparse, random networks of inhibitory neurons with nonlinear, conductance-based synapses. Emulations of these networks on the analog neuromorphic-hardware system Spikey allow us to test the efficiency of decorrelation by inhibitory feedback in the presence of hardware-specific heterogeneities. The configurability of the hardware substrate enables us to modulate the extent of heterogeneity in a systematic manner. We selectively study the effects of shared input and recurrent connections on correlations in membrane potentials and spike trains. Our results confirm that shared-input correlations are actively suppressed by inhibitory feedback also in highly heterogeneous networks exhibiting broad, heavy-tailed firing-rate distributions. In line with former studies, cell heterogeneities reduce shared-input correlations. Overall, however, correlations in the recurrent system can increase with the level of heterogeneity as a consequence of diminished effective negative feedback.

Clinical outcomes of liver transplantation for HBV-related hepatocellular carcinoma: data from the NIH HBV OLT study.

PubMed

Han, Steven-Huy; Reddy, K Rajender; Keeffe, Emmet B; Soldevila-Pico, Consuelo; Gish, Robert; Chung, Raymond T; Degertekin, Bulent; Lok, Anna

2011-01-01

Hepatitis B virus (HBV)-related hepatocellular carcinoma (HCC) is an indication for orthotopic liver transplantation (OLT) in patients with tumor stage within the United Network for Organ Sharing criteria. The number of patients listed for HBV-related HCC is increasing, while the number of patients listed for HBV-related cirrhosis is declining presumptively because of the availability of more effective oral nucleos(t)ide analogues. This study presents the final, long-term outcome of patients transplanted for HBV-related HCC in the National Institutes of Health (NIH) HBV OLT Study Group. Ninety-eight patients (52.4%) in the NIH HBV OLT cohort underwent OLT for HBV-related HCC. With a mean follow-up of 36.5 months post-OLT, 12 (12.2%) patients developed recurrence of HCC. Multivariate analysis did not find a statistically significant role of gender, tumor stage at OLT, pre-OLT HCC treatment, recurrence of HBV, or duration of HCC diagnosis pre-OLT in predicting HCC recurrence. Serum alpha-fetoprotein (AFP) level >200 ng/mL at transplant was found to be statistically significant in predicting HCC recurrence (p=0.003). HCC recurrence was significantly associated with decreased post-OLT survival. HCC is the most common indication for OLT in patients with chronic hepatitis B in the era of more effective oral antivirals. Serum AFP at the time of OLT is significantly associated with HCC recurrence. © 2010 John Wiley & Sons A/S.

A 15-gene signature for prediction of colon cancer recurrence and prognosis based on SVM.

PubMed

Xu, Guangru; Zhang, Minghui; Zhu, Hongxing; Xu, Jinhua

2017-03-10

To screen the gene signature for distinguishing patients with high risks from those with low-risks for colon cancer recurrence and predicting their prognosis. Five microarray datasets of colon cancer samples were collected from Gene Expression Omnibus database and one was obtained from The Cancer Genome Atlas (TCGA). After preprocessing, data in GSE17537 were analyzed using the Linear Models for Microarray data (LIMMA) method to identify the differentially expressed genes (DEGs). The DEGs further underwent PPI network-based neighborhood scoring and support vector machine (SVM) analyses to screen the feature genes associated with recurrence and prognosis, which were then validated by four datasets GSE38832, GSE17538, GSE28814 and TCGA using SVM and Cox regression analyses. A total of 1207 genes were identified as DEGs between recurrence and no-recurrence samples, including 726 downregulated and 481 upregulated genes. Using SVM analysis and five gene expression profile data confirmation, a 15-gene signature (HES5, ZNF417, GLRA2, OR8D2, HOXA7, FABP6, MUSK, HTR6, GRIP2, KLRK1, VEGFA, AKAP12, RHEB, NCRNA00152 and PMEPA1) were identified as a predictor of recurrence risk and prognosis for colon cancer patients. Our identified 15-gene signature may be useful to classify colon cancer patients with different prognosis and some genes in this signature may represent new therapeutic targets. Copyright © 2016. Published by Elsevier B.V.

Brain Dynamics in Predicting Driving Fatigue Using a Recurrent Self-Evolving Fuzzy Neural Network.

PubMed

Liu, Yu-Ting; Lin, Yang-Yin; Wu, Shang-Lin; Chuang, Chun-Hsiang; Lin, Chin-Teng

2016-02-01

This paper proposes a generalized prediction system called a recurrent self-evolving fuzzy neural network (RSEFNN) that employs an on-line gradient descent learning rule to address the electroencephalography (EEG) regression problem in brain dynamics for driving fatigue. The cognitive states of drivers significantly affect driving safety; in particular, fatigue driving, or drowsy driving, endangers both the individual and the public. For this reason, the development of brain-computer interfaces (BCIs) that can identify drowsy driving states is a crucial and urgent topic of study. Many EEG-based BCIs have been developed as artificial auxiliary systems for use in various practical applications because of the benefits of measuring EEG signals. In the literature, the efficacy of EEG-based BCIs in recognition tasks has been limited by low resolutions. The system proposed in this paper represents the first attempt to use the recurrent fuzzy neural network (RFNN) architecture to increase adaptability in realistic EEG applications to overcome this bottleneck. This paper further analyzes brain dynamics in a simulated car driving task in a virtual-reality environment. The proposed RSEFNN model is evaluated using the generalized cross-subject approach, and the results indicate that the RSEFNN is superior to competing models regardless of the use of recurrent or nonrecurrent structures.

Optimizing a neural network for detection of moving vehicles in video

NASA Astrophysics Data System (ADS)

Fischer, Noëlle M.; Kruithof, Maarten C.; Bouma, Henri

2017-10-01

In the field of security and defense, it is extremely important to reliably detect moving objects, such as cars, ships, drones and missiles. Detection and analysis of moving objects in cameras near borders could be helpful to reduce illicit trading, drug trafficking, irregular border crossing, trafficking in human beings and smuggling. Many recent benchmarks have shown that convolutional neural networks are performing well in the detection of objects in images. Most deep-learning research effort focuses on classification or detection on single images. However, the detection of dynamic changes (e.g., moving objects, actions and events) in streaming video is extremely relevant for surveillance and forensic applications. In this paper, we combine an end-to-end feedforward neural network for static detection with a recurrent Long Short-Term Memory (LSTM) network for multi-frame analysis. We present a practical guide with special attention to the selection of the optimizer and batch size. The end-to-end network is able to localize and recognize the vehicles in video from traffic cameras. We show an efficient way to collect relevant in-domain data for training with minimal manual labor. Our results show that the combination with LSTM improves performance for the detection of moving vehicles.

Global Synchronization of Multiple Recurrent Neural Networks With Time Delays via Impulsive Interactions.

PubMed

Yang, Shaofu; Guo, Zhenyuan; Wang, Jun

2017-07-01

In this paper, new results on the global synchronization of multiple recurrent neural networks (NNs) with time delays via impulsive interactions are presented. Impulsive interaction means that a number of NNs communicate with each other at impulse instants only, while they are independent at the remaining time. The communication topology among NNs is not required to be always connected and can switch ON and OFF at different impulse instants. By using the concept of sequential connectivity and the properties of stochastic matrices, a set of sufficient conditions depending on time delays is derived to ascertain global synchronization of multiple continuous-time recurrent NNs. In addition, a counterpart on the global synchronization of multiple discrete-time NNs is also discussed. Finally, two examples are presented to illustrate the results.

Characterization of dynamical systems under noise using recurrence networks: Application to simulated and EEG data

NASA Astrophysics Data System (ADS)

Puthanmadam Subramaniyam, Narayan; Hyttinen, Jari

2014-10-01

In this letter, we study the influence of observational noise on recurrence network (RN) measures, the global clustering coefficient (C) and average path length (L) using the Rössler system and propose the application of RN measures to analyze the structural properties of electroencephalographic (EEG) data. We find that for an appropriate recurrence rate (RR>0.02) the influence of noise on C can be minimized while L is independent of RR for increasing levels of noise. Indications of structural complexity were found for healthy EEG, but to a lesser extent than epileptic EEG. Furthermore, C performed better than L in case of epileptic EEG. Our results show that RN measures can provide insights into the structural properties of EEG in normal and pathological states.

Nonlinear Motion Tracking by Deep Learning Architecture

NASA Astrophysics Data System (ADS)

Verma, Arnav; Samaiya, Devesh; Gupta, Karunesh K.

2018-03-01

In the world of Artificial Intelligence, object motion tracking is one of the major problems. The extensive research is being carried out to track people in crowd. This paper presents a unique technique for nonlinear motion tracking in the absence of prior knowledge of nature of nonlinear path that the object being tracked may follow. We achieve this by first obtaining the centroid of the object and then using the centroid as the current example for a recurrent neural network trained using real-time recurrent learning. We have tweaked the standard algorithm slightly and have accumulated the gradient for few previous iterations instead of using just the current iteration as is the norm. We show that for a single object, such a recurrent neural network is highly capable of approximating the nonlinearity of its path.

Investigation of Back-off Based Interpolation Between Recurrent Neural Network and N-gram Language Models (Author’s Manuscript)

DTIC Science & Technology

2016-02-11

INVESTIGATION OF BACK-OFF BASED INTERPOLATION BETWEEN RECURRENT NEURAL NETWORK AND N- GRAM LANGUAGE MODELS X. Chen, X. Liu, M. J. F. Gales, and P. C...As the gener- alization patterns of RNNLMs and n- gram LMs are inherently dif- ferent, RNNLMs are usually combined with n- gram LMs via a fixed...RNNLMs and n- gram LMs as n- gram level changes. In order to fully exploit the detailed n- gram level comple- mentary attributes between the two LMs, a

Recall Performance for Content-Addressable Memory Using Adiabatic Quantum Optimization

DOE Office of Scientific and Technical Information (OSTI.GOV)

Imam, Neena; Humble, Travis S.; McCaskey, Alex

A content-addressable memory (CAM) stores key-value associations such that the key is recalled by providing its associated value. While CAM recall is traditionally performed using recurrent neural network models, we show how to solve this problem using adiabatic quantum optimization. Our approach maps the recurrent neural network to a commercially available quantum processing unit by taking advantage of the common underlying Ising spin model. We then assess the accuracy of the quantum processor to store key-value associations by quantifying recall performance against an ensemble of problem sets. We observe that different learning rules from the neural network community influence recallmore » accuracy but performance appears to be limited by potential noise in the processor. The strong connection established between quantum processors and neural network problems supports the growing intersection of these two ideas.« less

Convolutional neural networks for prostate cancer recurrence prediction

NASA Astrophysics Data System (ADS)

Kumar, Neeraj; Verma, Ruchika; Arora, Ashish; Kumar, Abhay; Gupta, Sanchit; Sethi, Amit; Gann, Peter H.

2017-03-01

Accurate prediction of the treatment outcome is important for cancer treatment planning. We present an approach to predict prostate cancer (PCa) recurrence after radical prostatectomy using tissue images. We used a cohort whose case vs. control (recurrent vs. non-recurrent) status had been determined using post-treatment follow up. Further, to aid the development of novel biomarkers of PCa recurrence, cases and controls were paired based on matching of other predictive clinical variables such as Gleason grade, stage, age, and race. For this cohort, tissue resection microarray with up to four cores per patient was available. The proposed approach is based on deep learning, and its novelty lies in the use of two separate convolutional neural networks (CNNs) - one to detect individual nuclei even in the crowded areas, and the other to classify them. To detect nuclear centers in an image, the first CNN predicts distance transform of the underlying (but unknown) multi-nuclear map from the input HE image. The second CNN classifies the patches centered at nuclear centers into those belonging to cases or controls. Voting across patches extracted from image(s) of a patient yields the probability of recurrence for the patient. The proposed approach gave 0.81 AUC for a sample of 30 recurrent cases and 30 non-recurrent controls, after being trained on an independent set of 80 case-controls pairs. If validated further, such an approach might help in choosing between a combination of treatment options such as active surveillance, radical prostatectomy, radiation, and hormone therapy. It can also generalize to the prediction of treatment outcomes in other cancers.

Suppression of anomalous synchronization and nonstationary behavior of neural network under small-world topology

NASA Astrophysics Data System (ADS)

Boaretto, B. R. R.; Budzinski, R. C.; Prado, T. L.; Kurths, J.; Lopes, S. R.

2018-05-01

It is known that neural networks under small-world topology can present anomalous synchronization and nonstationary behavior for weak coupling regimes. Here, we propose methods to suppress the anomalous synchronization and also to diminish the nonstationary behavior occurring in weakly coupled neural network under small-world topology. We consider a network of 2000 thermally sensitive identical neurons, based on the model of Hodgkin-Huxley in a small-world topology, with the probability of adding non local connection equal to p = 0 . 001. Based on experimental protocols to suppress anomalous synchronization, as well as nonstationary behavior of the neural network dynamics, we make use of (i) external stimulus (pulsed current); (ii) biologic parameters changing (neuron membrane conductance changes); and (iii) body temperature changes. Quantification analysis to evaluate phase synchronization makes use of the Kuramoto's order parameter, while recurrence quantification analysis, particularly the determinism, computed over the easily accessible mean field of network, the local field potential (LFP), is used to evaluate nonstationary states. We show that the methods proposed can control the anomalous synchronization and nonstationarity occurring for weak coupling parameter without any effect on the individual neuron dynamics, neither in the expected asymptotic synchronized states occurring for large values of the coupling parameter.

Intrahost Evolution of Methicillin-Resistant Staphylococcus aureus USA300 Among Individuals With Reoccurring Skin and Soft-Tissue Infections

PubMed Central

Azarian, Taj; Daum, Robert S.; Petty, Lindsay A.; Steinbeck, Jenny L.; Yin, Zachary; Nolan, David; Boyle-Vavra, Susan; Hanage, W. P.; Salemi, Marco; David, Michael Z.

2016-01-01

Background. Methicillin-resistant Staphylococcus aureus (MRSA) USA300 is the leading cause of MRSA infections in the United States and has caused an epidemic of skin and soft-tissue infections. Recurrent infections with USA300 MRSA are common, yet intrahost evolution during persistence on an individual has not been studied. This gap hinders the ability to clinically manage recurrent infections and reconstruct transmission networks. Methods. To characterize bacterial intrahost evolution, we examined the clinical courses of 4 subjects with 3–6 recurrent USA300 MRSA infections, using patient clinical data, including antibiotic exposure history, and whole-genome sequencing and phylogenetic analysis of all available MRSA isolates (n = 29). Results. Among sequential isolates, we found variability in diversity, accumulation of mutations, and mobile genetic elements. Selection for antimicrobial-resistant populations was observed through both an increase in the number of plasmids conferring multidrug resistance and strain replacement by a resistant population. Two of 4 subjects had strain replacement with a genetically distinct USA300 MRSA population. Discussions. During a 5-year period in 4 subjects, we identified development of antimicrobial resistance, intrahost evolution, and strain replacement among isolates from patients with recurrent MRSA infections. This calls into question the efficacy of decolonization to prevent recurrent infections and highlights the adaptive potential of USA300 and the need for effective sampling. PMID:27288537

Probability and volume of potential postwildfire debris flows in the 2012 High Park Burn Area near Fort Collins, Colorado

USGS Publications Warehouse

Verdin, Kristine L.; Dupree, Jean A.; Elliott, John G.

2012-01-01

This report presents a preliminary emergency assessment of the debris-flow hazards from drainage basins burned by the 2012 High Park fire near Fort Collins in Larimer County, Colorado. Empirical models derived from statistical evaluation of data collected from recently burned basins throughout the intermountain western United States were used to estimate the probability of debris-flow occurrence and volume of debris flows along the burned area drainage network and to estimate the same for 44 selected drainage basins along State Highway 14 and the perimeter of the burned area. Input data for the models included topographic parameters, soil characteristics, burn severity, and rainfall totals and intensities for a (1) 2-year-recurrence, 1-hour-duration rainfall (25 millimeters); (2) 10-year-recurrence, 1-hour-duration rainfall (43 millimeters); and (3) 25-year-recurrence, 1-hour-duration rainfall (51 millimeters). Estimated debris-flow probabilities along the drainage network and throughout the drainage basins of interest ranged from 1 to 84 percent in response to the 2-year-recurrence, 1-hour-duration rainfall; from 2 to 95 percent in response to the 10-year-recurrence, 1-hour-duration rainfall; and from 3 to 97 in response to the 25-year-recurrence, 1-hour-duration rainfall. Basins and drainage networks with the highest probabilities tended to be those on the eastern edge of the burn area where soils have relatively high clay contents and gradients are steep. Estimated debris-flow volumes range from a low of 1,600 cubic meters to a high of greater than 100,000 cubic meters. Estimated debris-flow volumes increase with basin size and distance along the drainage network, but some smaller drainages were also predicted to produce substantial volumes of material. The predicted probabilities and some of the volumes predicted for the modeled storms indicate a potential for substantial debris-flow impacts on structures, roads, bridges, and culverts located both within and immediately downstream from the burned area. Colorado State Highway 14 is also susceptible to impacts from debris flows.

Recurrent myocardial infarction: Mechanisms of free-floating adaptation and autonomic derangement in networked cardiac neural control

PubMed Central

Ardell, Jeffrey L.; Shivkumar, Kalyanam; Armour, J. Andrew

2017-01-01

The cardiac nervous system continuously controls cardiac function whether or not pathology is present. While myocardial infarction typically has a major and catastrophic impact, population studies have shown that longer-term risk for recurrent myocardial infarction and the related potential for sudden cardiac death depends mainly upon standard atherosclerotic variables and autonomic nervous system maladaptations. Investigative neurocardiology has demonstrated that autonomic control of cardiac function includes local circuit neurons for networked control within the peripheral nervous system. The structural and adaptive characteristics of such networked interactions define the dynamics and a new normal for cardiac control that results in the aftermath of recurrent myocardial infarction and/or unstable angina that may or may not precipitate autonomic derangement. These features are explored here via a mathematical model of cardiac regulation. A main observation is that the control environment during pathology is an extrapolation to a setting outside prior experience. Although global bounds guarantee stability, the resulting closed-loop dynamics exhibited while the network adapts during pathology are aptly described as ‘free-floating’ in order to emphasize their dependence upon details of the network structure. The totality of the results provide a mechanistic reasoning that validates the clinical practice of reducing sympathetic efferent neuronal tone while aggressively targeting autonomic derangement in the treatment of ischemic heart disease. PMID:28692680

Fitting of dynamic recurrent neural network models to sensory stimulus-response data.

PubMed

Doruk, R Ozgur; Zhang, Kechen

2018-06-02

We present a theoretical study aiming at model fitting for sensory neurons. Conventional neural network training approaches are not applicable to this problem due to lack of continuous data. Although the stimulus can be considered as a smooth time-dependent variable, the associated response will be a set of neural spike timings (roughly the instants of successive action potential peaks) that have no amplitude information. A recurrent neural network model can be fitted to such a stimulus-response data pair by using the maximum likelihood estimation method where the likelihood function is derived from Poisson statistics of neural spiking. The universal approximation feature of the recurrent dynamical neuron network models allows us to describe excitatory-inhibitory characteristics of an actual sensory neural network with any desired number of neurons. The stimulus data are generated by a phased cosine Fourier series having a fixed amplitude and frequency but a randomly shot phase. Various values of amplitude, stimulus component size, and sample size are applied in order to examine the effect of the stimulus to the identification process. Results are presented in tabular and graphical forms at the end of this text. In addition, to demonstrate the success of this research, a study involving the same model, nominal parameters and stimulus structure, and another study that works on different models are compared to that of this research.

Cost-effectiveness of Apixaban Versus Other Oral Anticoagulants for the Initial Treatment of Venous Thromboembolism and Prevention of Recurrence.

PubMed

Lanitis, Tereza; Leipold, Robert; Hamilton, Melissa; Rublee, Dale; Quon, Peter; Browne, Chantelle; Cohen, Alexander T

2016-03-01

To assess the cost-effectiveness of apixaban versus rivaroxaban, low-molecular-weight heparin (LMWH)/dabigatran, and LMWH/vitamin K antagonist (VKA) for the initial treatment and prevention of recurrent thromboembolic events in patients with venous thromboembolism (VTE). A Markov model was developed to evaluate the pharmacoeconomic effect of 6 months of treatment with apixaban versus other anticoagulants over a lifetime horizon. Network meta-analyses were conducted using the results of the Apixaban after the Initial Management of Pulmonary Embolism and Deep Vein Thrombosis with First-Line Therapy (AMPLIFY), EINSTEIN-pooled, and RE-COVER I and II trials for the following end points: recurrent VTE, major bleeds, clinically relevant non-major bleeds, and treatment discontinuations. The analysis was conducted from the perspective of the United Kingdom National Health Service. The outcomes evaluated were the number of events avoided in a 1000-patient cohort, total costs, life years, quality-adjusted life years (QALYs), and cost per QALY gained over a patient's lifetime. Treatment for 6 months with apixaban was projected to result in fewer recurrent VTE and bleeding events in comparison to rivaroxaban, LMWH/dabigatran, and LMWH/VKA. Apixaban was cost-effective compared with LMWH/VKA at an incremental cost-effectiveness ratio of £2520 per QALY gained and was a dominant (ie, lower costs and higher QALYs) alternative to either rivaroxaban or LMWH/dabigatran. Sensitivity analysis indicated that results were robust over a wide range of inputs. The assessment of the effects and costs of apixaban in this study predicted that apixaban is a dominant alternative to rivaroxaban and LMWH/dabigatran and a cost-effective alternative to LMWH/VKA for 6 months of treatment of VTE and the prevention of recurrence. Copyright © 2016 Elsevier HS Journals, Inc. All rights reserved.

Self-Organization of Microcircuits in Networks of Spiking Neurons with Plastic Synapses.

PubMed

Ocker, Gabriel Koch; Litwin-Kumar, Ashok; Doiron, Brent

2015-08-01

The synaptic connectivity of cortical networks features an overrepresentation of certain wiring motifs compared to simple random-network models. This structure is shaped, in part, by synaptic plasticity that promotes or suppresses connections between neurons depending on their joint spiking activity. Frequently, theoretical studies focus on how feedforward inputs drive plasticity to create this network structure. We study the complementary scenario of self-organized structure in a recurrent network, with spike timing-dependent plasticity driven by spontaneous dynamics. We develop a self-consistent theory for the evolution of network structure by combining fast spiking covariance with a slow evolution of synaptic weights. Through a finite-size expansion of network dynamics we obtain a low-dimensional set of nonlinear differential equations for the evolution of two-synapse connectivity motifs. With this theory in hand, we explore how the form of the plasticity rule drives the evolution of microcircuits in cortical networks. When potentiation and depression are in approximate balance, synaptic dynamics depend on weighted divergent, convergent, and chain motifs. For additive, Hebbian STDP these motif interactions create instabilities in synaptic dynamics that either promote or suppress the initial network structure. Our work provides a consistent theoretical framework for studying how spiking activity in recurrent networks interacts with synaptic plasticity to determine network structure.

Self-Organization of Microcircuits in Networks of Spiking Neurons with Plastic Synapses

PubMed Central

Ocker, Gabriel Koch; Litwin-Kumar, Ashok; Doiron, Brent

2015-01-01

The synaptic connectivity of cortical networks features an overrepresentation of certain wiring motifs compared to simple random-network models. This structure is shaped, in part, by synaptic plasticity that promotes or suppresses connections between neurons depending on their joint spiking activity. Frequently, theoretical studies focus on how feedforward inputs drive plasticity to create this network structure. We study the complementary scenario of self-organized structure in a recurrent network, with spike timing-dependent plasticity driven by spontaneous dynamics. We develop a self-consistent theory for the evolution of network structure by combining fast spiking covariance with a slow evolution of synaptic weights. Through a finite-size expansion of network dynamics we obtain a low-dimensional set of nonlinear differential equations for the evolution of two-synapse connectivity motifs. With this theory in hand, we explore how the form of the plasticity rule drives the evolution of microcircuits in cortical networks. When potentiation and depression are in approximate balance, synaptic dynamics depend on weighted divergent, convergent, and chain motifs. For additive, Hebbian STDP these motif interactions create instabilities in synaptic dynamics that either promote or suppress the initial network structure. Our work provides a consistent theoretical framework for studying how spiking activity in recurrent networks interacts with synaptic plasticity to determine network structure. PMID:26291697

Modular, Hierarchical Learning By Artificial Neural Networks

NASA Technical Reports Server (NTRS)

Baldi, Pierre F.; Toomarian, Nikzad

1996-01-01

Modular and hierarchical approach to supervised learning by artificial neural networks leads to neural networks more structured than neural networks in which all neurons fully interconnected. These networks utilize general feedforward flow of information and sparse recurrent connections to achieve dynamical effects. The modular organization, sparsity of modular units and connections, and fact that learning is much more circumscribed are all attractive features for designing neural-network hardware. Learning streamlined by imitating some aspects of biological neural networks.

Coexpression network analysis identifies transcriptional modules associated with genomic alterations in neuroblastoma.

PubMed

Yang, Liulin; Li, Yun; Wei, Zhi; Chang, Xiao

2018-06-01

Neuroblastoma is a highly complex and heterogeneous cancer in children. Acquired genomic alterations including MYCN amplification, 1p deletion and 11q deletion are important risk factors and biomarkers in neuroblastoma. Here, we performed a co-expression-based gene network analysis to study the intrinsic association between specific genomic changes and transcriptome organization. We identified multiple gene coexpression modules which are recurrent in two independent datasets and associated with functional pathways including nervous system development, cell cycle, immune system process and extracellular matrix/space. Our results also indicated that modules involved in nervous system development and cell cycle are highly associated with MYCN amplification and 1p deletion, while modules responding to immune system process are associated with MYCN amplification only. In summary, this integrated analysis provides novel insights into molecular heterogeneity and pathogenesis of neuroblastoma. This article is part of a Special Issue entitled: Accelerating Precision Medicine through Genetic and Genomic Big Data Analysis edited by Yudong Cai & Tao Huang. Copyright © 2017. Published by Elsevier B.V.

Mechanisms of Seizure Propagation in 2-Dimensional Centre-Surround Recurrent Networks

PubMed Central

Hall, David; Kuhlmann, Levin

2013-01-01

Understanding how seizures spread throughout the brain is an important problem in the treatment of epilepsy, especially for implantable devices that aim to avert focal seizures before they spread to, and overwhelm, the rest of the brain. This paper presents an analysis of the speed of propagation in a computational model of seizure-like activity in a 2-dimensional recurrent network of integrate-and-fire neurons containing both excitatory and inhibitory populations and having a difference of Gaussians connectivity structure, an approximation to that observed in cerebral cortex. In the same computational model network, alternative mechanisms are explored in order to simulate the range of seizure-like activity propagation speeds (0.1–100 mm/s) observed in two animal-slice-based models of epilepsy: (1) low extracellular , which creates excess excitation and (2) introduction of gamma-aminobutyric acid (GABA) antagonists, which reduce inhibition. Moreover, two alternative connection topologies are considered: excitation broader than inhibition, and inhibition broader than excitation. It was found that the empirically observed range of propagation velocities can be obtained for both connection topologies. For the case of the GABA antagonist model simulation, consistent with other studies, it was found that there is an effective threshold in the degree of inhibition below which waves begin to propagate. For the case of the low extracellular model simulation, it was found that activity-dependent reductions in inhibition provide a potential explanation for the emergence of slowly propagating waves. This was simulated as a depression of inhibitory synapses, but it may also be achieved by other mechanisms. This work provides a localised network understanding of the propagation of seizures in 2-dimensional centre-surround networks that can be tested empirically. PMID:23967201

Effects of Calcium Spikes in the Layer 5 Pyramidal Neuron on Coincidence Detection and Activity Propagation

PubMed Central

Chua, Yansong; Morrison, Abigail

2016-01-01

The role of dendritic spiking mechanisms in neural processing is so far poorly understood. To investigate the role of calcium spikes in the functional properties of the single neuron and recurrent networks, we investigated a three compartment neuron model of the layer 5 pyramidal neuron with calcium dynamics in the distal compartment. By performing single neuron simulations with noisy synaptic input and occasional large coincident input at either just the distal compartment or at both somatic and distal compartments, we show that the presence of calcium spikes confers a substantial advantage for coincidence detection in the former case and a lesser advantage in the latter. We further show that the experimentally observed critical frequency phenomenon, in which action potentials triggered by stimuli near the soma above a certain frequency trigger a calcium spike at distal dendrites, leading to further somatic depolarization, is not exhibited by a neuron receiving realistically noisy synaptic input, and so is unlikely to be a necessary component of coincidence detection. We next investigate the effect of calcium spikes in propagation of spiking activities in a feed-forward network (FFN) embedded in a balanced recurrent network. The excitatory neurons in the network are again connected to either just the distal, or both somatic and distal compartments. With purely distal connectivity, activity propagation is stable and distinguishable for a large range of recurrent synaptic strengths if the feed-forward connections are sufficiently strong, but propagation does not occur in the absence of calcium spikes. When connections are made to both the somatic and the distal compartments, activity propagation is achieved for neurons with active calcium dynamics at a much smaller number of neurons per pool, compared to a network of passive neurons, but quickly becomes unstable as the strength of recurrent synapses increases. Activity propagation at higher scaling factors can be stabilized by increasing network inhibition or introducing short term depression in the excitatory synapses, but the signal to noise ratio remains low. Our results demonstrate that the interaction of synchrony with dendritic spiking mechanisms can have profound consequences for the dynamics on the single neuron and network level. PMID:27499740

Effects of Calcium Spikes in the Layer 5 Pyramidal Neuron on Coincidence Detection and Activity Propagation.

PubMed

Chua, Yansong; Morrison, Abigail

2016-01-01

The role of dendritic spiking mechanisms in neural processing is so far poorly understood. To investigate the role of calcium spikes in the functional properties of the single neuron and recurrent networks, we investigated a three compartment neuron model of the layer 5 pyramidal neuron with calcium dynamics in the distal compartment. By performing single neuron simulations with noisy synaptic input and occasional large coincident input at either just the distal compartment or at both somatic and distal compartments, we show that the presence of calcium spikes confers a substantial advantage for coincidence detection in the former case and a lesser advantage in the latter. We further show that the experimentally observed critical frequency phenomenon, in which action potentials triggered by stimuli near the soma above a certain frequency trigger a calcium spike at distal dendrites, leading to further somatic depolarization, is not exhibited by a neuron receiving realistically noisy synaptic input, and so is unlikely to be a necessary component of coincidence detection. We next investigate the effect of calcium spikes in propagation of spiking activities in a feed-forward network (FFN) embedded in a balanced recurrent network. The excitatory neurons in the network are again connected to either just the distal, or both somatic and distal compartments. With purely distal connectivity, activity propagation is stable and distinguishable for a large range of recurrent synaptic strengths if the feed-forward connections are sufficiently strong, but propagation does not occur in the absence of calcium spikes. When connections are made to both the somatic and the distal compartments, activity propagation is achieved for neurons with active calcium dynamics at a much smaller number of neurons per pool, compared to a network of passive neurons, but quickly becomes unstable as the strength of recurrent synapses increases. Activity propagation at higher scaling factors can be stabilized by increasing network inhibition or introducing short term depression in the excitatory synapses, but the signal to noise ratio remains low. Our results demonstrate that the interaction of synchrony with dendritic spiking mechanisms can have profound consequences for the dynamics on the single neuron and network level.

Reinforced two-step-ahead weight adjustment technique for online training of recurrent neural networks.

PubMed

Chang, Li-Chiu; Chen, Pin-An; Chang, Fi-John

2012-08-01

A reliable forecast of future events possesses great value. The main purpose of this paper is to propose an innovative learning technique for reinforcing the accuracy of two-step-ahead (2SA) forecasts. The real-time recurrent learning (RTRL) algorithm for recurrent neural networks (RNNs) can effectively model the dynamics of complex processes and has been used successfully in one-step-ahead forecasts for various time series. A reinforced RTRL algorithm for 2SA forecasts using RNNs is proposed in this paper, and its performance is investigated by two famous benchmark time series and a streamflow during flood events in Taiwan. Results demonstrate that the proposed reinforced 2SA RTRL algorithm for RNNs can adequately forecast the benchmark (theoretical) time series, significantly improve the accuracy of flood forecasts, and effectively reduce time-lag effects.

When do correlations increase with firing rates in recurrent networks?

PubMed Central

2017-01-01

A central question in neuroscience is to understand how noisy firing patterns are used to transmit information. Because neural spiking is noisy, spiking patterns are often quantified via pairwise correlations, or the probability that two cells will spike coincidentally, above and beyond their baseline firing rate. One observation frequently made in experiments, is that correlations can increase systematically with firing rate. Theoretical studies have determined that stimulus-dependent correlations that increase with firing rate can have beneficial effects on information coding; however, we still have an incomplete understanding of what circuit mechanisms do, or do not, produce this correlation-firing rate relationship. Here, we studied the relationship between pairwise correlations and firing rates in recurrently coupled excitatory-inhibitory spiking networks with conductance-based synapses. We found that with stronger excitatory coupling, a positive relationship emerged between pairwise correlations and firing rates. To explain these findings, we used linear response theory to predict the full correlation matrix and to decompose correlations in terms of graph motifs. We then used this decomposition to explain why covariation of correlations with firing rate—a relationship previously explained in feedforward networks driven by correlated input—emerges in some recurrent networks but not in others. Furthermore, when correlations covary with firing rate, this relationship is reflected in low-rank structure in the correlation matrix. PMID:28448499

Learning and retrieval behavior in recurrent neural networks with pre-synaptic dependent homeostatic plasticity

NASA Astrophysics Data System (ADS)

Mizusaki, Beatriz E. P.; Agnes, Everton J.; Erichsen, Rubem; Brunnet, Leonardo G.

2017-08-01

The plastic character of brain synapses is considered to be one of the foundations for the formation of memories. There are numerous kinds of such phenomenon currently described in the literature, but their role in the development of information pathways in neural networks with recurrent architectures is still not completely clear. In this paper we study the role of an activity-based process, called pre-synaptic dependent homeostatic scaling, in the organization of networks that yield precise-timed spiking patterns. It encodes spatio-temporal information in the synaptic weights as it associates a learned input with a specific response. We introduce a correlation measure to evaluate the precision of the spiking patterns and explore the effects of different inhibitory interactions and learning parameters. We find that large learning periods are important in order to improve the network learning capacity and discuss this ability in the presence of distinct inhibitory currents.

A novel word spotting method based on recurrent neural networks.

PubMed

Frinken, Volkmar; Fischer, Andreas; Manmatha, R; Bunke, Horst

2012-02-01

Keyword spotting refers to the process of retrieving all instances of a given keyword from a document. In the present paper, a novel keyword spotting method for handwritten documents is described. It is derived from a neural network-based system for unconstrained handwriting recognition. As such it performs template-free spotting, i.e., it is not necessary for a keyword to appear in the training set. The keyword spotting is done using a modification of the CTC Token Passing algorithm in conjunction with a recurrent neural network. We demonstrate that the proposed systems outperform not only a classical dynamic time warping-based approach but also a modern keyword spotting system, based on hidden Markov models. Furthermore, we analyze the performance of the underlying neural networks when using them in a recognition task followed by keyword spotting on the produced transcription. We point out the advantages of keyword spotting when compared to classic text line recognition.

An Asynchronous Recurrent Network of Cellular Automaton-Based Neurons and Its Reproduction of Spiking Neural Network Activities.

PubMed

Matsubara, Takashi; Torikai, Hiroyuki

2016-04-01

Modeling and implementation approaches for the reproduction of input-output relationships in biological nervous tissues contribute to the development of engineering and clinical applications. However, because of high nonlinearity, the traditional modeling and implementation approaches encounter difficulties in terms of generalization ability (i.e., performance when reproducing an unknown data set) and computational resources (i.e., computation time and circuit elements). To overcome these difficulties, asynchronous cellular automaton-based neuron (ACAN) models, which are described as special kinds of cellular automata that can be implemented as small asynchronous sequential logic circuits have been proposed. This paper presents a novel type of such ACAN and a theoretical analysis of its excitability. This paper also presents a novel network of such neurons, which can mimic input-output relationships of biological and nonlinear ordinary differential equation model neural networks. Numerical analyses confirm that the presented network has a higher generalization ability than other major modeling and implementation approaches. In addition, Field-Programmable Gate Array-implementations confirm that the presented network requires lower computational resources.

Identification of eight candidate target genes of the recurrent 3p12-p14 loss in cervical cancer by integrative genomic profiling.

PubMed

Lando, Malin; Wilting, Saskia M; Snipstad, Kristin; Clancy, Trevor; Bierkens, Mariska; Aarnes, Eva-Katrine; Holden, Marit; Stokke, Trond; Sundfør, Kolbein; Holm, Ruth; Kristensen, Gunnar B; Steenbergen, Renske D M; Lyng, Heidi

2013-05-01

The pathogenetic role, including its target genes, of the recurrent 3p12-p14 loss in cervical cancer has remained unclear. To determine the onset of the event during carcinogenesis, we used microarray techniques and found that the loss was the most frequent 3p event, occurring in 61% of 92 invasive carcinomas, in only 2% of 43 high-grade intraepithelial lesions (CIN2/3), and in 33% of 6 CIN3 lesions adjacent to invasive carcinomas, suggesting a role in acquisition of invasiveness or early during the invasive phase. We performed an integrative DNA copy number and expression analysis of 77 invasive carcinomas, where all genes within the recurrent region were included. We selected eight genes, THOC7, PSMD6, SLC25A26, TMF1, RYBP, SHQ1, EBLN2, and GBE1, which were highly down-regulated in cases with loss, as confirmed at the protein level for RYBP and TMF1 by immunohistochemistry. The eight genes were subjected to network analysis based on the expression profiles, revealing interaction partners of proteins encoded by the genes that were coordinately regulated in tumours with loss. Several partners were shared among the eight genes, indicating crosstalk in their signalling. Gene ontology analysis showed enrichment of biological processes such as apoptosis, proliferation, and stress response in the network and suggested a relationship between down-regulation of the eight genes and activation of tumourigenic pathways. Survival analysis showed prognostic impact of the eight-gene signature that was confirmed in a validation cohort of 74 patients and was independent of clinical parameters. These results support the role of the eight candidate genes as targets of the 3p12-p14 loss in cervical cancer and suggest that the strong selection advantage of the loss during carcinogenesis might be caused by a synergetic effect of several tumourigenic processes controlled by these targets. Copyright © 2013 Pathological Society of Great Britain and Ireland. Published by John Wiley & Sons, Ltd.

New baseline correction algorithm for text-line recognition with bidirectional recurrent neural networks

NASA Astrophysics Data System (ADS)

Morillot, Olivier; Likforman-Sulem, Laurence; Grosicki, Emmanuèle

2013-04-01

Many preprocessing techniques have been proposed for isolated word recognition. However, recently, recognition systems have dealt with text blocks and their compound text lines. In this paper, we propose a new preprocessing approach to efficiently correct baseline skew and fluctuations. Our approach is based on a sliding window within which the vertical position of the baseline is estimated. Segmentation of text lines into subparts is, thus, avoided. Experiments conducted on a large publicly available database (Rimes), with a BLSTM (bidirectional long short-term memory) recurrent neural network recognition system, show that our baseline correction approach highly improves performance.

INDIRECT INTELLIGENT SLIDING MODE CONTROL OF A SHAPE MEMORY ALLOY ACTUATED FLEXIBLE BEAM USING HYSTERETIC RECURRENT NEURAL NETWORKS.

PubMed

Hannen, Jennifer C; Crews, John H; Buckner, Gregory D

2012-08-01

This paper introduces an indirect intelligent sliding mode controller (IISMC) for shape memory alloy (SMA) actuators, specifically a flexible beam deflected by a single offset SMA tendon. The controller manipulates applied voltage, which alters SMA tendon temperature to track reference bending angles. A hysteretic recurrent neural network (HRNN) captures the nonlinear, hysteretic relationship between SMA temperature and bending angle. The variable structure control strategy provides robustness to model uncertainties and parameter variations, while effectively compensating for system nonlinearities, achieving superior tracking compared to an optimized PI controller.

Automatic construction of a recurrent neural network based classifier for vehicle passage detection

NASA Astrophysics Data System (ADS)

Burnaev, Evgeny; Koptelov, Ivan; Novikov, German; Khanipov, Timur

2017-03-01

Recurrent Neural Networks (RNNs) are extensively used for time-series modeling and prediction. We propose an approach for automatic construction of a binary classifier based on Long Short-Term Memory RNNs (LSTM-RNNs) for detection of a vehicle passage through a checkpoint. As an input to the classifier we use multidimensional signals of various sensors that are installed on the checkpoint. Obtained results demonstrate that the previous approach to handcrafting a classifier, consisting of a set of deterministic rules, can be successfully replaced by an automatic RNN training on an appropriately labelled data.

Prefrontal cortical network activity: Opposite effects of psychedelic hallucinogens and D1/D5 dopamine receptor activation

PubMed Central

Lambe, Evelyn K.; Aghajanian, George K.

2007-01-01

The fine-tuning of network activity provides a modulating influence on how information is processed and interpreted in the brain. Here, we use brain slices of rat prefrontal cortex to study how recurrent network activity is affected by neuromodulators known to alter normal cortical function. We previously determined that glutamate spillover and stimulation of extrasynaptic NMDA receptors are required to support hallucinogen-induced cortical network activity. Since microdialysis studies suggest that psychedelic hallucinogens and dopamine D1/D5 receptor agonists have opposite effects on extracellular glutamate in prefrontal cortex, we hypothesized that these two families of psychoactive drugs would have opposite effects on cortical network activity. We found that network activity can be enhanced by DOI (a psychedelic hallucinogen that is a partial agonist of serotonin 5-HT2A/2C receptors) and suppressed by the selective D1/D5 agonist SKF 38393. This suppression could be mimicked by direct activation of adenylyl cyclase with forskolin or by addition of a cAMP analog. These findings are consistent with previous work showing that activation of adenylyl cyclase can upregulate neuronal glutamate transporters, thereby decreasing synaptic spillover of glutamate. Consistent with this hypothesis, a low concentration of the glutamate transporter inhibitor TBOA restored electrically-evoked recurrent activity in the presence of a selective D1/D5 agonist, whereas recurrent activity in the presence of a low level of the GABAA antagonist bicuculline was not resistant to suppression by the D1/D5 agonist. The tempering of network UP states by D1/D5 receptor activation may have implications for the proposed use of D1/D5 agonists in the treatment of schizophrenia. PMID:17293055

Local community detection as pattern restoration by attractor dynamics of recurrent neural networks.

PubMed

Okamoto, Hiroshi

2016-08-01

Densely connected parts in networks are referred to as "communities". Community structure is a hallmark of a variety of real-world networks. Individual communities in networks form functional modules of complex systems described by networks. Therefore, finding communities in networks is essential to approaching and understanding complex systems described by networks. In fact, network science has made a great deal of effort to develop effective and efficient methods for detecting communities in networks. Here we put forward a type of community detection, which has been little examined so far but will be practically useful. Suppose that we are given a set of source nodes that includes some (but not all) of "true" members of a particular community; suppose also that the set includes some nodes that are not the members of this community (i.e., "false" members of the community). We propose to detect the community from this "imperfect" and "inaccurate" set of source nodes using attractor dynamics of recurrent neural networks. Community detection by the proposed method can be viewed as restoration of the original pattern from a deteriorated pattern, which is analogous to cue-triggered recall of short-term memory in the brain. We demonstrate the effectiveness of the proposed method using synthetic networks and real social networks for which correct communities are known. Copyright © 2016 Elsevier Ireland Ltd. All rights reserved.

Exact solutions for rate and synchrony in recurrent networks of coincidence detectors.

PubMed

Mikula, Shawn; Niebur, Ernst

2008-11-01

We provide analytical solutions for mean firing rates and cross-correlations of coincidence detector neurons in recurrent networks with excitatory or inhibitory connectivity, with rate-modulated steady-state spiking inputs. We use discrete-time finite-state Markov chains to represent network state transition probabilities, which are subsequently used to derive exact analytical solutions for mean firing rates and cross-correlations. As illustrated in several examples, the method can be used for modeling cortical microcircuits and clarifying single-neuron and population coding mechanisms. We also demonstrate that increasing firing rates do not necessarily translate into increasing cross-correlations, though our results do support the contention that firing rates and cross-correlations are likely to be coupled. Our analytical solutions underscore the complexity of the relationship between firing rates and cross-correlations.

Improving protein disorder prediction by deep bidirectional long short-term memory recurrent neural networks.

PubMed

Hanson, Jack; Yang, Yuedong; Paliwal, Kuldip; Zhou, Yaoqi

2017-03-01

Capturing long-range interactions between structural but not sequence neighbors of proteins is a long-standing challenging problem in bioinformatics. Recently, long short-term memory (LSTM) networks have significantly improved the accuracy of speech and image classification problems by remembering useful past information in long sequential events. Here, we have implemented deep bidirectional LSTM recurrent neural networks in the problem of protein intrinsic disorder prediction. The new method, named SPOT-Disorder, has steadily improved over a similar method using a traditional, window-based neural network (SPINE-D) in all datasets tested without separate training on short and long disordered regions. Independent tests on four other datasets including the datasets from critical assessment of structure prediction (CASP) techniques and >10 000 annotated proteins from MobiDB, confirmed SPOT-Disorder as one of the best methods in disorder prediction. Moreover, initial studies indicate that the method is more accurate in predicting functional sites in disordered regions. These results highlight the usefulness combining LSTM with deep bidirectional recurrent neural networks in capturing non-local, long-range interactions for bioinformatics applications. SPOT-disorder is available as a web server and as a standalone program at: http://sparks-lab.org/server/SPOT-disorder/index.php . j.hanson@griffith.edu.au or yuedong.yang@griffith.edu.au or yaoqi.zhou@griffith.edu.au. Supplementary data is available at Bioinformatics online. © The Author 2016. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com

Decreased Resting-State Activity in the Precuneus Is Associated With Depressive Episodes in Recurrent Depression.

PubMed

Liu, Chun-Hong; Ma, Xin; Yuan, Zhen; Song, Lu-Ping; Jing, Bing; Lu, Hong-Yu; Tang, Li-Rong; Fan, Jin; Walter, Martin; Liu, Cun-Zhi; Wang, Lihong; Wang, Chuan-Yue

2017-04-01

To investigate alterations in resting-state spontaneous brain activity in patients with major depressive disorder (MDD) experiencing multiple episodes. Between May 2007 and September 2014, 24 recurrent and 22 remitted patients diagnosed with MDD with the Structured Clinical Interview for DSM-IV Axis I Disorders (SCID-I), and 69 healthy controls matched for age, sex, and educational level participated in this study. Among them, 1 healthy control was excluded due to excessive head motion. The fractional amplitude of low-frequency fluctuation (fALFF) was assessed for all recruited subjects during the completion of resting-state functional magnetic resonance imaging. Relationships between fALFF and clinical measurements, including number of depressive episodes and illness duration, were examined. Compared to patients with remitted MDD and to healthy controls, patients with recurrent MDD exhibited decreased fALFF in the right posterior insula and right precuneus and increased fALFF in the left ventral anterior cingulate cortex. Decreased fALFF in the right precuneus and increased fALFF in the right middle insula were correlated with the number of depressive episodes in the recurrent MDD groups (r = -0.75, P < .01 and r = 0.78, P < .01, respectively) and remitted MDD groups (r = -0.63, P < .01 and r = 0.41, P = .03, respectively). In addition to regions in the default mode network (DMN) and salience network, the altered resting-state activity in the middle temporal and visual cortices was also identified. Altered resting-state activity was observed across several neural networks in patients with recurrent MDD. Consistent with the emerging theory that altered DMN activity is a risk factor for depression relapses, the association between reduced fALFF in the right precuneus and number of depressive episodes supports the role of the DMN in the pathology of recurrent depression. © Copyright 2017 Physicians Postgraduate Press, Inc.

Connectomics-based analysis of information flow in the Drosophila brain.

PubMed

Shih, Chi-Tin; Sporns, Olaf; Yuan, Shou-Li; Su, Ta-Shun; Lin, Yen-Jen; Chuang, Chao-Chun; Wang, Ting-Yuan; Lo, Chung-Chuang; Greenspan, Ralph J; Chiang, Ann-Shyn

2015-05-18

Understanding the overall patterns of information flow within the brain has become a major goal of neuroscience. In the current study, we produced a first draft of the Drosophila connectome at the mesoscopic scale, reconstructed from 12,995 images of neuron projections collected in FlyCircuit (version 1.1). Neuron polarities were predicted according to morphological criteria, with nodes of the network corresponding to brain regions designated as local processing units (LPUs). The weight of each directed edge linking a pair of LPUs was determined by the number of neuron terminals that connected one LPU to the other. The resulting network showed hierarchical structure and small-world characteristics and consisted of five functional modules that corresponded to sensory modalities (olfactory, mechanoauditory, and two visual) and the pre-motor center. Rich-club organization was present in this network and involved LPUs in all sensory centers, and rich-club members formed a putative motor center of the brain. Major intra- and inter-modular loops were also identified that could play important roles for recurrent and reverberant information flow. The present analysis revealed whole-brain patterns of network structure and information flow. Additionally, we propose that the overall organizational scheme showed fundamental similarities to the network structure of the mammalian brain. Copyright © 2015 Elsevier Ltd. All rights reserved.

Non—Linear Flood Assessment with Neural Network

NASA Astrophysics Data System (ADS)

Murariu, Gabriel; Puscasu, Gheorghe; Gogoncea, Vlad

2010-01-01

In our days, theoretical investigations are used in obtaining the mathematical model for the studied systems or processes. In general, the dynamics of the system are deeply nonlinear, complex or unknown. Generally speaking, such complex structure is a set of interconnected components. The common approach is therefore to start from measurements of the behavior of the system and the external influences (inputs) and try to determine a mathematical relation between them without going into the details of what is actually happening inside the system. Such strategy had known a great success during the time and it was applied for a large class of multifaceted processes. Accepting this approach, there could be investigated the climatic phenomena. In this paper is presented, in a comparative way, a non-linear water flood assessment made in a very sensitive area of the Lower Danube zone where, in the past years, a series of climatic problems have been happening. In these conditions, climatic risk factor management is a necessity. In a regular way, there could be considered and designed nonlinear models for the climatic factors' analysis by using a huge historical evidence data archive. In a previous paper we reached a notable intermediary result basing on a mathematical model constructed on internal recurrent neural network structure. Such approach had been presented considering the internal state estimation when no measurements coming from the sensors are available for system states. A modified backpropagation algorithm had been introduced in order to train the internal recurrent neural networks for nonlinear system identification. In this paper is exposed a comparative study between a numerical advances based on fluid dynamics' equations and our previous approach, based on internal recurrent neural networks (IRNN). The numerical approaching was made in order to succeed in building a physics model of a water flow evaluation and further, to achieve including the rainfall contributions. This condition is necessary for prediction and it is the first step toward a DSS—Decision Support System in the area. The relationship between the simulated results and the registered data allows considering our particular method to be useful for considered water flood assessment.

Construction of Gene Regulatory Networks Using Recurrent Neural Networks and Swarm Intelligence.

PubMed

Khan, Abhinandan; Mandal, Sudip; Pal, Rajat Kumar; Saha, Goutam

2016-01-01

We have proposed a methodology for the reverse engineering of biologically plausible gene regulatory networks from temporal genetic expression data. We have used established information and the fundamental mathematical theory for this purpose. We have employed the Recurrent Neural Network formalism to extract the underlying dynamics present in the time series expression data accurately. We have introduced a new hybrid swarm intelligence framework for the accurate training of the model parameters. The proposed methodology has been first applied to a small artificial network, and the results obtained suggest that it can produce the best results available in the contemporary literature, to the best of our knowledge. Subsequently, we have implemented our proposed framework on experimental (in vivo) datasets. Finally, we have investigated two medium sized genetic networks (in silico) extracted from GeneNetWeaver, to understand how the proposed algorithm scales up with network size. Additionally, we have implemented our proposed algorithm with half the number of time points. The results indicate that a reduction of 50% in the number of time points does not have an effect on the accuracy of the proposed methodology significantly, with a maximum of just over 15% deterioration in the worst case.

Intrahost Evolution of Methicillin-Resistant Staphylococcus aureus USA300 Among Individuals With Reoccurring Skin and Soft-Tissue Infections.

PubMed

Azarian, Taj; Daum, Robert S; Petty, Lindsay A; Steinbeck, Jenny L; Yin, Zachary; Nolan, David; Boyle-Vavra, Susan; Hanage, W P; Salemi, Marco; David, Michael Z

2016-09-15

Methicillin-resistant Staphylococcus aureus (MRSA) USA300 is the leading cause of MRSA infections in the United States and has caused an epidemic of skin and soft-tissue infections. Recurrent infections with USA300 MRSA are common, yet intrahost evolution during persistence on an individual has not been studied. This gap hinders the ability to clinically manage recurrent infections and reconstruct transmission networks. To characterize bacterial intrahost evolution, we examined the clinical courses of 4 subjects with 3-6 recurrent USA300 MRSA infections, using patient clinical data, including antibiotic exposure history, and whole-genome sequencing and phylogenetic analysis of all available MRSA isolates (n = 29). Among sequential isolates, we found variability in diversity, accumulation of mutations, and mobile genetic elements. Selection for antimicrobial-resistant populations was observed through both an increase in the number of plasmids conferring multidrug resistance and strain replacement by a resistant population. Two of 4 subjects had strain replacement with a genetically distinct USA300 MRSA population. During a 5-year period in 4 subjects, we identified development of antimicrobial resistance, intrahost evolution, and strain replacement among isolates from patients with recurrent MRSA infections. This calls into question the efficacy of decolonization to prevent recurrent infections and highlights the adaptive potential of USA300 and the need for effective sampling. © The Author 2016. Published by Oxford University Press for the Infectious Diseases Society of America. All rights reserved. For permissions, e-mail journals.permissions@oup.com.

Oral antiviral therapy for prevention of genital herpes outbreaks in immunocompetent and nonpregnant patients.

PubMed

Le Cleach, Laurence; Trinquart, Ludovic; Do, Giao; Maruani, Annabel; Lebrun-Vignes, Benedicte; Ravaud, Philippe; Chosidow, Olivier

2014-08-03

Genital herpes is caused by herpes simplex virus 1 (HSV-1) or 2 (HSV-2). Some infected people experience outbreaks of genital herpes, typically, characterized by vesicular and erosive localized painful genital lesions. To compare the effectiveness and safety of three oral antiviral drugs (acyclovir, famciclovir and valacyclovir) prescribed to suppress genital herpes outbreaks in non-pregnant patients. We searched the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, EMBASE, the search portal of the World Health Organization International Clinical Trials Registry Platform and pharmaceutical company databases up to February 2014. We also searched US Food and Drug Administration databases and proceedings of seven congresses to a maximum of 10 years. We contacted trial authors and pharmaceutical companies. We selected parallel-group and cross-over randomized controlled trials including patients with recurrent genital herpes caused by HSV, whatever the type (HSV-1, HSV-2, or undetermined), with at least four recurrences per year (trials concerning human immunodeficiency virus (HIV)-positive patients or pregnant women were not eligible) and comparing suppressive oral antiviral treatment with oral acyclovir, famciclovir, and valacyclovir versus placebo or another suppressive oral antiviral treatment. Two review authors independently selected eligible trials and extracted data. The Risk of bias tool was used to assess risk of bias. Treatment effect was measured by the risk ratio (RR) of having at least one genital herpes recurrence. Pooled RRs were derived by conventional pairwise meta-analyses. A network meta-analysis allowed for estimation of all possible two-by-two comparisons between antiviral drugs. A total of 26 trials (among which six had a cross-over design) were included. Among the 6950 randomly assigned participants, 54% (range 0 to 100%) were female, mean age was 35 years (range 26 to 45.1), and the mean number of recurrences per year was 11 (range 6.3 to 17.8). Duration of treatment was two to 12 months. Risk of bias was considered high for half of the studies and unclear for the other half. A total of 14 trials compared acyclovir versus placebo, four trials compared valacyclovir versus placebo and 2 trials compared valacyclovir versus no treatment. Three trials compared famciclovir versus placebo. Two trials compared valacyclovir versus famciclovir and one trial compared acyclovir versus valacyclovir versus placebo.We analyzed data from 22 trials for the outcome: risk of having at least one clinical recurrence. We could not obtain the outcome data for four trials. In placebo-controlled trials, there was a low quality evidence that the risk of having at least one clinical recurrence was reduced with acyclovir (nine parallel-group trials, n = 2049; pooled RR 0.48, 95% confidence interval (CI) 0.39 to 0.58), valacyclovir (four trials, n = 1788; pooled RR 0.41, 95% CI 0.24 to 0.69), or famciclovir (two trials, n = 732; pooled RR 0.57, 95% CI 0.50 to 0.64). The six cross-over trials showed larger treatment effects on average than the parallel-group trials. We found evidence of a small-study effect for acyclovir placebo-controlled trials (adjusted pooled RR 0.61, 95% CI 0.49 to 0.75). In analyzing parallel-group trials by daily dose, no clear evidence was found of a dose-response relationship for any drug. In head-to-head trials, the risk of having at least one recurrence was increased with valacyclovir rather than acyclovir (one trial, n = 1345; RR 1.16, 95% CI 1.01 to 1.34) and was not significantly different from that seen with famciclovir as compared with valacyclovir (one trial, n = 320; RR 1.18, 95% CI 0.86 to 1.63).We included 16 parallel-arm trials in a network meta-analysis and we were unable to determine which of the drugs was most effective in reducing the risk of at least one clinical recurrence (after adjustment for small-study effects, pooled RR 0.83, 95% CI 0.61 to 1.11 for valacyclovir vs acyclovir; pooled RR 1.04, 95% CI, 0.71 to 1.49 for famciclovir vs acyclovir; and pooled RR 1.26, 95% CI 0.89 to 1.75 for famciclovir vs valacyclovir). Safety data were sought but were reported as total numbers of adverse events. Owing to risk of bias and inconsistency, there is low quality evidence that suppressive antiviral therapy with acyclovir, valacyclovir or famciclovir in pacients experiencing at least four recurrences of genital herpes per year decreases the number of pacients with at least one recurrence as compared with placebo. Network meta-analysis of the few direct comparisons and the indirect comparisons did not show superiority of one drug over another.

A system of recurrent neural networks for modularising, parameterising and dynamic analysis of cell signalling networks.

PubMed

Samarasinghe, S; Ling, H

In this paper, we show how to extend our previously proposed novel continuous time Recurrent Neural Networks (RNN) approach that retains the advantage of continuous dynamics offered by Ordinary Differential Equations (ODE) while enabling parameter estimation through adaptation, to larger signalling networks using a modular approach. Specifically, the signalling network is decomposed into several sub-models based on important temporal events in the network. Each sub-model is represented by the proposed RNN and trained using data generated from the corresponding ODE model. Trained sub-models are assembled into a whole system RNN which is then subjected to systems dynamics and sensitivity analyses. The concept is illustrated by application to G1/S transition in cell cycle using Iwamoto et al. (2008) ODE model. We decomposed the G1/S network into 3 sub-models: (i) E2F transcription factor release; (ii) E2F and CycE positive feedback loop for elevating cyclin levels; and (iii) E2F and CycA negative feedback to degrade E2F. The trained sub-models accurately represented system dynamics and parameters were in good agreement with the ODE model. The whole system RNN however revealed couple of parameters contributing to compounding errors due to feedback and required refinement to sub-model 2. These related to the reversible reaction between CycE/CDK2 and p27, its inhibitor. The revised whole system RNN model very accurately matched dynamics of the ODE system. Local sensitivity analysis of the whole system model further revealed the most dominant influence of the above two parameters in perturbing G1/S transition, giving support to a recent hypothesis that the release of inhibitor p27 from Cyc/CDK complex triggers cell cycle stage transition. To make the model useful in a practical setting, we modified each RNN sub-model with a time relay switch to facilitate larger interval input data (≈20min) (original model used data for 30s or less) and retrained them that produced parameters and protein concentrations similar to the original RNN system. Results thus demonstrated the reliability of the proposed RNN method for modelling relatively large networks by modularisation for practical settings. Advantages of the method are its ability to represent accurate continuous system dynamics and ease of: parameter estimation through training with data from a practical setting, model analysis (40% faster than ODE), fine tuning parameters when more data are available, sub-model extension when new elements and/or interactions come to light and model expansion with addition of sub-models. Copyright © 2017 Elsevier B.V. All rights reserved.

The Stillbirth Classification System for the Safe Passage Study: Incorporating Mechanism, Etiology, and Recurrence

PubMed Central

Boyd, Theonia K.; Wright, Colleen A.; Odendaal, Hein J.; Elliott, Amy J.; Sens, Mary Ann; Folkerth, Rebecca D.; Roberts, Drucilla J.; Kinney, Hannah C.

2017-01-01

OBJECTIVE Describe the classification system for the assignment of the cause of death for stillbirth in the Safe Passage Study, an international, multi-institutional, prospective analysis conducted by the NIAAA/NICHD funded PASS Network (The Prenatal Alcohol in SIDS and Stillbirth (PASS) Research Network). The study mission is to determine the role of prenatal alcohol and/or cigarette smoke exposure in adverse pregnancy outcomes, including stillbirth, in a high-risk cohort of 12,000 maternal/fetal dyads. METHODS The PASS Network classification system is based upon 5 ‘sites of origin’ for cause of stillbirth (Fetal, Placental, Maternal, External/Environmental, or Undetermined), further subdivided into mechanism subcategories (e.g., Placental Perfusion Failure). Both site of origin and mechanism stratification are employed to assign an ultimate cause of death. Each PASS stillbirth (n=19) in the feasibility study was assigned a cause of death, and status of sporadic versus recurrent. Adjudication involved review of the maternal and obstetrical records, and fetal autopsy and placental findings, with complete consensus in each case. Two published classification systems, i.e., INCODE and ReCoDe, were used for comparison. RESULTS Causes of stillbirth classified were: fetal (n=5, 26%), placental (n=10, 53%), external (n=1, 5%), and undetermined (n=3, 16%). Nine cases (47%) had placental causes of death due to maternal disorders that carry recurrence risks. There was complete agreement for the cause of death across the three classification systems in 26% of cases, and a combination of partial or complete agreement in 68% of cases. Complete vs. partial agreements were predicated upon the classification schemes used for comparison. CONCLUSIONS The proposed PASS system is a user-friendly classification system that provides comparable information to previously published systems. Advantages include its simplicity, mechanistic formulations, tight clinicopathologic integration, provision for an undetermined category, and its wide applicability for use by perinatal mortality review boards with access to information routinely collected during clinicopathologic evaluations. PMID:27116324

Genomic analyses identify molecular subtypes of pancreatic cancer.

PubMed

Bailey, Peter; Chang, David K; Nones, Katia; Johns, Amber L; Patch, Ann-Marie; Gingras, Marie-Claude; Miller, David K; Christ, Angelika N; Bruxner, Tim J C; Quinn, Michael C; Nourse, Craig; Murtaugh, L Charles; Harliwong, Ivon; Idrisoglu, Senel; Manning, Suzanne; Nourbakhsh, Ehsan; Wani, Shivangi; Fink, Lynn; Holmes, Oliver; Chin, Venessa; Anderson, Matthew J; Kazakoff, Stephen; Leonard, Conrad; Newell, Felicity; Waddell, Nick; Wood, Scott; Xu, Qinying; Wilson, Peter J; Cloonan, Nicole; Kassahn, Karin S; Taylor, Darrin; Quek, Kelly; Robertson, Alan; Pantano, Lorena; Mincarelli, Laura; Sanchez, Luis N; Evers, Lisa; Wu, Jianmin; Pinese, Mark; Cowley, Mark J; Jones, Marc D; Colvin, Emily K; Nagrial, Adnan M; Humphrey, Emily S; Chantrill, Lorraine A; Mawson, Amanda; Humphris, Jeremy; Chou, Angela; Pajic, Marina; Scarlett, Christopher J; Pinho, Andreia V; Giry-Laterriere, Marc; Rooman, Ilse; Samra, Jaswinder S; Kench, James G; Lovell, Jessica A; Merrett, Neil D; Toon, Christopher W; Epari, Krishna; Nguyen, Nam Q; Barbour, Andrew; Zeps, Nikolajs; Moran-Jones, Kim; Jamieson, Nigel B; Graham, Janet S; Duthie, Fraser; Oien, Karin; Hair, Jane; Grützmann, Robert; Maitra, Anirban; Iacobuzio-Donahue, Christine A; Wolfgang, Christopher L; Morgan, Richard A; Lawlor, Rita T; Corbo, Vincenzo; Bassi, Claudio; Rusev, Borislav; Capelli, Paola; Salvia, Roberto; Tortora, Giampaolo; Mukhopadhyay, Debabrata; Petersen, Gloria M; Munzy, Donna M; Fisher, William E; Karim, Saadia A; Eshleman, James R; Hruban, Ralph H; Pilarsky, Christian; Morton, Jennifer P; Sansom, Owen J; Scarpa, Aldo; Musgrove, Elizabeth A; Bailey, Ulla-Maja Hagbo; Hofmann, Oliver; Sutherland, Robert L; Wheeler, David A; Gill, Anthony J; Gibbs, Richard A; Pearson, John V; Waddell, Nicola; Biankin, Andrew V; Grimmond, Sean M

2016-03-03

Integrated genomic analysis of 456 pancreatic ductal adenocarcinomas identified 32 recurrently mutated genes that aggregate into 10 pathways: KRAS, TGF-β, WNT, NOTCH, ROBO/SLIT signalling, G1/S transition, SWI-SNF, chromatin modification, DNA repair and RNA processing. Expression analysis defined 4 subtypes: (1) squamous; (2) pancreatic progenitor; (3) immunogenic; and (4) aberrantly differentiated endocrine exocrine (ADEX) that correlate with histopathological characteristics. Squamous tumours are enriched for TP53 and KDM6A mutations, upregulation of the TP63∆N transcriptional network, hypermethylation of pancreatic endodermal cell-fate determining genes and have a poor prognosis. Pancreatic progenitor tumours preferentially express genes involved in early pancreatic development (FOXA2/3, PDX1 and MNX1). ADEX tumours displayed upregulation of genes that regulate networks involved in KRAS activation, exocrine (NR5A2 and RBPJL), and endocrine differentiation (NEUROD1 and NKX2-2). Immunogenic tumours contained upregulated immune networks including pathways involved in acquired immune suppression. These data infer differences in the molecular evolution of pancreatic cancer subtypes and identify opportunities for therapeutic development.

Processing speed in recurrent visual networks correlates with general intelligence.

PubMed

Jolij, Jacob; Huisman, Danielle; Scholte, Steven; Hamel, Ronald; Kemner, Chantal; Lamme, Victor A F

2007-01-08

Studies on the neural basis of general fluid intelligence strongly suggest that a smarter brain processes information faster. Different brain areas, however, are interconnected by both feedforward and feedback projections. Whether both types of connections or only one of the two types are faster in smarter brains remains unclear. Here we show, by measuring visual evoked potentials during a texture discrimination task, that general fluid intelligence shows a strong correlation with processing speed in recurrent visual networks, while there is no correlation with speed of feedforward connections. The hypothesis that a smarter brain runs faster may need to be refined: a smarter brain's feedback connections run faster.

Stability of discrete time recurrent neural networks and nonlinear optimization problems.

PubMed

Singh, Jayant; Barabanov, Nikita

2016-02-01

We consider the method of Reduction of Dissipativity Domain to prove global Lyapunov stability of Discrete Time Recurrent Neural Networks. The standard and advanced criteria for Absolute Stability of these essentially nonlinear systems produce rather weak results. The method mentioned above is proved to be more powerful. It involves a multi-step procedure with maximization of special nonconvex functions over polytopes on every step. We derive conditions which guarantee an existence of at most one point of local maximum for such functions over every hyperplane. This nontrivial result is valid for wide range of neuron transfer functions. Copyright © 2015 Elsevier Ltd. All rights reserved.

Identification of serial number on bank card using recurrent neural network

NASA Astrophysics Data System (ADS)

Liu, Li; Huang, Linlin; Xue, Jian

2018-04-01

Identification of serial number on bank card has many applications. Due to the different number printing mode, complex background, distortion in shape, etc., it is quite challenging to achieve high identification accuracy. In this paper, we propose a method using Normalization-Cooperated Gradient Feature (NCGF) and Recurrent Neural Network (RNN) based on Long Short-Term Memory (LSTM) for serial number identification. The NCGF maps the gradient direction elements of original image to direction planes such that the RNN with direction planes as input can recognize numbers more accurately. Taking the advantages of NCGF and RNN, we get 90%digit string recognition accuracy.

Deep recurrent neural network reveals a hierarchy of process memory during dynamic natural vision.

PubMed

Shi, Junxing; Wen, Haiguang; Zhang, Yizhen; Han, Kuan; Liu, Zhongming

2018-05-01

The human visual cortex extracts both spatial and temporal visual features to support perception and guide behavior. Deep convolutional neural networks (CNNs) provide a computational framework to model cortical representation and organization for spatial visual processing, but unable to explain how the brain processes temporal information. To overcome this limitation, we extended a CNN by adding recurrent connections to different layers of the CNN to allow spatial representations to be remembered and accumulated over time. The extended model, or the recurrent neural network (RNN), embodied a hierarchical and distributed model of process memory as an integral part of visual processing. Unlike the CNN, the RNN learned spatiotemporal features from videos to enable action recognition. The RNN better predicted cortical responses to natural movie stimuli than the CNN, at all visual areas, especially those along the dorsal stream. As a fully observable model of visual processing, the RNN also revealed a cortical hierarchy of temporal receptive window, dynamics of process memory, and spatiotemporal representations. These results support the hypothesis of process memory, and demonstrate the potential of using the RNN for in-depth computational understanding of dynamic natural vision. © 2018 Wiley Periodicals, Inc.

Lifelong learning of human actions with deep neural network self-organization.

PubMed

Parisi, German I; Tani, Jun; Weber, Cornelius; Wermter, Stefan

2017-12-01

Lifelong learning is fundamental in autonomous robotics for the acquisition and fine-tuning of knowledge through experience. However, conventional deep neural models for action recognition from videos do not account for lifelong learning but rather learn a batch of training data with a predefined number of action classes and samples. Thus, there is the need to develop learning systems with the ability to incrementally process available perceptual cues and to adapt their responses over time. We propose a self-organizing neural architecture for incrementally learning to classify human actions from video sequences. The architecture comprises growing self-organizing networks equipped with recurrent neurons for processing time-varying patterns. We use a set of hierarchically arranged recurrent networks for the unsupervised learning of action representations with increasingly large spatiotemporal receptive fields. Lifelong learning is achieved in terms of prediction-driven neural dynamics in which the growth and the adaptation of the recurrent networks are driven by their capability to reconstruct temporally ordered input sequences. Experimental results on a classification task using two action benchmark datasets show that our model is competitive with state-of-the-art methods for batch learning also when a significant number of sample labels are missing or corrupted during training sessions. Additional experiments show the ability of our model to adapt to non-stationary input avoiding catastrophic interference. Copyright © 2017 The Author(s). Published by Elsevier Ltd.. All rights reserved.

Speed of feedforward and recurrent processing in multilayer networks of integrate-and-fire neurons.

PubMed

Panzeri, S; Rolls, E T; Battaglia, F; Lavis, R

2001-11-01

The speed of processing in the visual cortical areas can be fast, with for example the latency of neuronal responses increasing by only approximately 10 ms per area in the ventral visual system sequence V1 to V2 to V4 to inferior temporal visual cortex. This has led to the suggestion that rapid visual processing can only be based on the feedforward connections between cortical areas. To test this idea, we investigated the dynamics of information retrieval in multiple layer networks using a four-stage feedforward network modelled with continuous dynamics with integrate-and-fire neurons, and associative synaptic connections between stages with a synaptic time constant of 10 ms. Through the implementation of continuous dynamics, we found latency differences in information retrieval of only 5 ms per layer when local excitation was absent and processing was purely feedforward. However, information latency differences increased significantly when non-associative local excitation was included. We also found that local recurrent excitation through associatively modified synapses can contribute significantly to processing in as little as 15 ms per layer, including the feedforward and local feedback processing. Moreover, and in contrast to purely feed-forward processing, the contribution of local recurrent feedback was useful and approximately this rapid even when retrieval was made difficult by noise. These findings suggest that cortical information processing can benefit from recurrent circuits when the allowed processing time per cortical area is at least 15 ms long.

A deep learning approach for real time prostate segmentation in freehand ultrasound guided biopsy.

PubMed

Anas, Emran Mohammad Abu; Mousavi, Parvin; Abolmaesumi, Purang

2018-06-01

Targeted prostate biopsy, incorporating multi-parametric magnetic resonance imaging (mp-MRI) and its registration with ultrasound, is currently the state-of-the-art in prostate cancer diagnosis. The registration process in most targeted biopsy systems today relies heavily on accurate segmentation of ultrasound images. Automatic or semi-automatic segmentation is typically performed offline prior to the start of the biopsy procedure. In this paper, we present a deep neural network based real-time prostate segmentation technique during the biopsy procedure, hence paving the way for dynamic registration of mp-MRI and ultrasound data. In addition to using convolutional networks for extracting spatial features, the proposed approach employs recurrent networks to exploit the temporal information among a series of ultrasound images. One of the key contributions in the architecture is to use residual convolution in the recurrent networks to improve optimization. We also exploit recurrent connections within and across different layers of the deep networks to maximize the utilization of the temporal information. Furthermore, we perform dense and sparse sampling of the input ultrasound sequence to make the network robust to ultrasound artifacts. Our architecture is trained on 2,238 labeled transrectal ultrasound images, with an additional 637 and 1,017 unseen images used for validation and testing, respectively. We obtain a mean Dice similarity coefficient of 93%, a mean surface distance error of 1.10 mm and a mean Hausdorff distance error of 3.0 mm. A comparison of the reported results with those of a state-of-the-art technique indicates statistically significant improvement achieved by the proposed approach. Copyright © 2018 Elsevier B.V. All rights reserved.

A Three-Threshold Learning Rule Approaches the Maximal Capacity of Recurrent Neural Networks

PubMed Central

Alemi, Alireza; Baldassi, Carlo; Brunel, Nicolas; Zecchina, Riccardo

2015-01-01

Understanding the theoretical foundations of how memories are encoded and retrieved in neural populations is a central challenge in neuroscience. A popular theoretical scenario for modeling memory function is the attractor neural network scenario, whose prototype is the Hopfield model. The model simplicity and the locality of the synaptic update rules come at the cost of a poor storage capacity, compared with the capacity achieved with perceptron learning algorithms. Here, by transforming the perceptron learning rule, we present an online learning rule for a recurrent neural network that achieves near-maximal storage capacity without an explicit supervisory error signal, relying only upon locally accessible information. The fully-connected network consists of excitatory binary neurons with plastic recurrent connections and non-plastic inhibitory feedback stabilizing the network dynamics; the memory patterns to be memorized are presented online as strong afferent currents, producing a bimodal distribution for the neuron synaptic inputs. Synapses corresponding to active inputs are modified as a function of the value of the local fields with respect to three thresholds. Above the highest threshold, and below the lowest threshold, no plasticity occurs. In between these two thresholds, potentiation/depression occurs when the local field is above/below an intermediate threshold. We simulated and analyzed a network of binary neurons implementing this rule and measured its storage capacity for different sizes of the basins of attraction. The storage capacity obtained through numerical simulations is shown to be close to the value predicted by analytical calculations. We also measured the dependence of capacity on the strength of external inputs. Finally, we quantified the statistics of the resulting synaptic connectivity matrix, and found that both the fraction of zero weight synapses and the degree of symmetry of the weight matrix increase with the number of stored patterns. PMID:26291608

A Three-Threshold Learning Rule Approaches the Maximal Capacity of Recurrent Neural Networks.

PubMed

Alemi, Alireza; Baldassi, Carlo; Brunel, Nicolas; Zecchina, Riccardo

2015-08-01

Understanding the theoretical foundations of how memories are encoded and retrieved in neural populations is a central challenge in neuroscience. A popular theoretical scenario for modeling memory function is the attractor neural network scenario, whose prototype is the Hopfield model. The model simplicity and the locality of the synaptic update rules come at the cost of a poor storage capacity, compared with the capacity achieved with perceptron learning algorithms. Here, by transforming the perceptron learning rule, we present an online learning rule for a recurrent neural network that achieves near-maximal storage capacity without an explicit supervisory error signal, relying only upon locally accessible information. The fully-connected network consists of excitatory binary neurons with plastic recurrent connections and non-plastic inhibitory feedback stabilizing the network dynamics; the memory patterns to be memorized are presented online as strong afferent currents, producing a bimodal distribution for the neuron synaptic inputs. Synapses corresponding to active inputs are modified as a function of the value of the local fields with respect to three thresholds. Above the highest threshold, and below the lowest threshold, no plasticity occurs. In between these two thresholds, potentiation/depression occurs when the local field is above/below an intermediate threshold. We simulated and analyzed a network of binary neurons implementing this rule and measured its storage capacity for different sizes of the basins of attraction. The storage capacity obtained through numerical simulations is shown to be close to the value predicted by analytical calculations. We also measured the dependence of capacity on the strength of external inputs. Finally, we quantified the statistics of the resulting synaptic connectivity matrix, and found that both the fraction of zero weight synapses and the degree of symmetry of the weight matrix increase with the number of stored patterns.

Intrinsically-generated fluctuating activity in excitatory-inhibitory networks.

PubMed

Mastrogiuseppe, Francesca; Ostojic, Srdjan

2017-04-01

Recurrent networks of non-linear units display a variety of dynamical regimes depending on the structure of their synaptic connectivity. A particularly remarkable phenomenon is the appearance of strongly fluctuating, chaotic activity in networks of deterministic, but randomly connected rate units. How this type of intrinsically generated fluctuations appears in more realistic networks of spiking neurons has been a long standing question. To ease the comparison between rate and spiking networks, recent works investigated the dynamical regimes of randomly-connected rate networks with segregated excitatory and inhibitory populations, and firing rates constrained to be positive. These works derived general dynamical mean field (DMF) equations describing the fluctuating dynamics, but solved these equations only in the case of purely inhibitory networks. Using a simplified excitatory-inhibitory architecture in which DMF equations are more easily tractable, here we show that the presence of excitation qualitatively modifies the fluctuating activity compared to purely inhibitory networks. In presence of excitation, intrinsically generated fluctuations induce a strong increase in mean firing rates, a phenomenon that is much weaker in purely inhibitory networks. Excitation moreover induces two different fluctuating regimes: for moderate overall coupling, recurrent inhibition is sufficient to stabilize fluctuations; for strong coupling, firing rates are stabilized solely by the upper bound imposed on activity, even if inhibition is stronger than excitation. These results extend to more general network architectures, and to rate networks receiving noisy inputs mimicking spiking activity. Finally, we show that signatures of the second dynamical regime appear in networks of integrate-and-fire neurons.

Intrinsically-generated fluctuating activity in excitatory-inhibitory networks

PubMed Central

Mastrogiuseppe, Francesca; Ostojic, Srdjan

2017-01-01

Recurrent networks of non-linear units display a variety of dynamical regimes depending on the structure of their synaptic connectivity. A particularly remarkable phenomenon is the appearance of strongly fluctuating, chaotic activity in networks of deterministic, but randomly connected rate units. How this type of intrinsically generated fluctuations appears in more realistic networks of spiking neurons has been a long standing question. To ease the comparison between rate and spiking networks, recent works investigated the dynamical regimes of randomly-connected rate networks with segregated excitatory and inhibitory populations, and firing rates constrained to be positive. These works derived general dynamical mean field (DMF) equations describing the fluctuating dynamics, but solved these equations only in the case of purely inhibitory networks. Using a simplified excitatory-inhibitory architecture in which DMF equations are more easily tractable, here we show that the presence of excitation qualitatively modifies the fluctuating activity compared to purely inhibitory networks. In presence of excitation, intrinsically generated fluctuations induce a strong increase in mean firing rates, a phenomenon that is much weaker in purely inhibitory networks. Excitation moreover induces two different fluctuating regimes: for moderate overall coupling, recurrent inhibition is sufficient to stabilize fluctuations; for strong coupling, firing rates are stabilized solely by the upper bound imposed on activity, even if inhibition is stronger than excitation. These results extend to more general network architectures, and to rate networks receiving noisy inputs mimicking spiking activity. Finally, we show that signatures of the second dynamical regime appear in networks of integrate-and-fire neurons. PMID:28437436

An early look at the Organ Procurement and Transplantation Network explant pathology form data.

PubMed

Harper, Ann M; Edwards, Erick; Washburn, W Kenneth; Heimbach, Julie

2016-06-01

In April 2012, the Organ Procurement and Transplantation Network (OPTN) implemented an online explant pathology form for recipients of liver transplantation who received additional wait-list priority for their diagnosis of hepatocellular carcinoma (HCC). The purpose of the form was to standardize the data being reported to the OPTN, which had been required since 2002 but were submitted to the OPTN in a variety of formats via facsimile. From April 2012 to December 2014, over 4500 explant forms were submitted, allowing for detailed analysis of the characteristics of the explanted livers. Data from the explant pathology forms were used to assess agreement with pretransplant imaging. Explant data were also used to assess the risk of recurrence. Of those with T2 priority, 55.7% were found to be stage T2 on explant. Extrahepatic spread (odds ratio [OR] = 6.8; P < 0.01), poor tumor differentiation (OR = 2.8; P < 0.01), microvascular invasion (OR = 2.6; P < 0.01), macrovascular invasion (OR = 3.2; P < 0.01), and whether the Milan stage based on the number and size of tumors on the explant form was T4 (OR = 2.4; P < 0.01) were the strongest predictors of recurrence. In conclusion, this analysis confirms earlier findings that showed an incomplete agreement between pretransplant imaging and posttransplant pathology in terms of HCC staging, though the number of patients with both no pretransplant treatment and no tumor in the explant was reduced from 20% to <1%. In addition, several factors were identified (eg, tumor burden, age, sex, region, ablative therapy, alpha-fetoprotein, Milan stage, vascular invasion, satellite lesions, etc.) that were predictive of HCC recurrence, allowing for more targeted surveillance of high-risk recipients. Continued evaluation of these data will help shape future guidelines or policy recommendations. Liver Transplantation 22 757-764 2016 AASLD. © 2016 American Association for the Study of Liver Diseases.

Recurrent neural network approach to quantum signal: coherent state restoration for continuous-variable quantum key distribution

NASA Astrophysics Data System (ADS)

Lu, Weizhao; Huang, Chunhui; Hou, Kun; Shi, Liting; Zhao, Huihui; Li, Zhengmei; Qiu, Jianfeng

2018-05-01

In continuous-variable quantum key distribution (CV-QKD), weak signal carrying information transmits from Alice to Bob; during this process it is easily influenced by unknown noise which reduces signal-to-noise ratio, and strongly impacts reliability and stability of the communication. Recurrent quantum neural network (RQNN) is an artificial neural network model which can perform stochastic filtering without any prior knowledge of the signal and noise. In this paper, a modified RQNN algorithm with expectation maximization algorithm is proposed to process the signal in CV-QKD, which follows the basic rule of quantum mechanics. After RQNN, noise power decreases about 15 dBm, coherent signal recognition rate of RQNN is 96%, quantum bit error rate (QBER) drops to 4%, which is 6.9% lower than original QBER, and channel capacity is notably enlarged.

Exact Solutions for Rate and Synchrony in Recurrent Networks of Coincidence Detectors

PubMed Central

Mikula, Shawn; Niebur, Ernst

2009-01-01

We provide analytical solutions for mean firing rates and cross-correlations of coincidence detector neurons in recurrent networks with excitatory or inhibitory connectivity with rate-modulated steady-state spiking inputs. We use discrete-time finite-state Markov chains to represent network state transition probabilities, which are subsequently used to derive exact analytical solutions for mean firing rates and cross-correlations. As illustrated in several examples, the method can be used for modeling cortical microcircuits and clarifying single-neuron and population coding mechanisms. We also demonstrate that increasing firing rates do not necessarily translate into increasing cross-correlations, though our results do support the contention that firing rates and cross-correlations are likely to be coupled. Our analytical solutions underscore the complexity of the relationship between firing rates and cross-correlations. PMID:18439133

Exact event-driven implementation for recurrent networks of stochastic perfect integrate-and-fire neurons.

PubMed

Taillefumier, Thibaud; Touboul, Jonathan; Magnasco, Marcelo

2012-12-01

In vivo cortical recording reveals that indirectly driven neural assemblies can produce reliable and temporally precise spiking patterns in response to stereotyped stimulation. This suggests that despite being fundamentally noisy, the collective activity of neurons conveys information through temporal coding. Stochastic integrate-and-fire models delineate a natural theoretical framework to study the interplay of intrinsic neural noise and spike timing precision. However, there are inherent difficulties in simulating their networks' dynamics in silico with standard numerical discretization schemes. Indeed, the well-posedness of the evolution of such networks requires temporally ordering every neuronal interaction, whereas the order of interactions is highly sensitive to the random variability of spiking times. Here, we answer these issues for perfect stochastic integrate-and-fire neurons by designing an exact event-driven algorithm for the simulation of recurrent networks, with delayed Dirac-like interactions. In addition to being exact from the mathematical standpoint, our proposed method is highly efficient numerically. We envision that our algorithm is especially indicated for studying the emergence of polychronized motifs in networks evolving under spike-timing-dependent plasticity with intrinsic noise.

Recurrent Neural Network Applications for Astronomical Time Series

NASA Astrophysics Data System (ADS)

Protopapas, Pavlos

2017-06-01

The benefits of good predictive models in astronomy lie in early event prediction systems and effective resource allocation. Current time series methods applicable to regular time series have not evolved to generalize for irregular time series. In this talk, I will describe two Recurrent Neural Network methods, Long Short-Term Memory (LSTM) and Echo State Networks (ESNs) for predicting irregular time series. Feature engineering along with a non-linear modeling proved to be an effective predictor. For noisy time series, the prediction is improved by training the network on error realizations using the error estimates from astronomical light curves. In addition to this, we propose a new neural network architecture to remove correlation from the residuals in order to improve prediction and compensate for the noisy data. Finally, I show how to set hyperparameters for a stable and performant solution correctly. In this work, we circumvent this obstacle by optimizing ESN hyperparameters using Bayesian optimization with Gaussian Process priors. This automates the tuning procedure, enabling users to employ the power of RNN without needing an in-depth understanding of the tuning procedure.

Co-expression network analysis identified six hub genes in association with metastasis risk and prognosis in hepatocellular carcinoma

PubMed Central

Feng, Juerong; Zhou, Rui; Chang, Ying; Liu, Jing; Zhao, Qiu

2017-01-01

Hepatocellular carcinoma (HCC) has a high incidence and mortality worldwide, and its carcinogenesis and progression are influenced by a complex network of gene interactions. A weighted gene co-expression network was constructed to identify gene modules associated with the clinical traits in HCC (n = 214). Among the 13 modules, high correlation was only found between the red module and metastasis risk (classified by the HCC metastasis gene signature) (R2 = −0.74). Moreover, in the red module, 34 network hub genes for metastasis risk were identified, six of which (ABAT, AGXT, ALDH6A1, CYP4A11, DAO and EHHADH) were also hub nodes in the protein-protein interaction network of the module genes. Thus, a total of six hub genes were identified. In validation, all hub genes showed a negative correlation with the four-stage HCC progression (P for trend < 0.05) in the test set. Furthermore, in the training set, HCC samples with any hub gene lowly expressed demonstrated a higher recurrence rate and poorer survival rate (hazard ratios with 95% confidence intervals > 1). RNA-sequencing data of 142 HCC samples showed consistent results in the prognosis. Gene set enrichment analysis (GSEA) demonstrated that in the samples with any hub gene highly expressed, a total of 24 functional gene sets were enriched, most of which focused on amino acid metabolism and oxidation. In conclusion, co-expression network analysis identified six hub genes in association with HCC metastasis risk and prognosis, which might improve the prognosis by influencing amino acid metabolism and oxidation. PMID:28430663

Object class segmentation of RGB-D video using recurrent convolutional neural networks.

PubMed

Pavel, Mircea Serban; Schulz, Hannes; Behnke, Sven

2017-04-01

Object class segmentation is a computer vision task which requires labeling each pixel of an image with the class of the object it belongs to. Deep convolutional neural networks (DNN) are able to learn and take advantage of local spatial correlations required for this task. They are, however, restricted by their small, fixed-sized filters, which limits their ability to learn long-range dependencies. Recurrent Neural Networks (RNN), on the other hand, do not suffer from this restriction. Their iterative interpretation allows them to model long-range dependencies by propagating activity. This property is especially useful when labeling video sequences, where both spatial and temporal long-range dependencies occur. In this work, a novel RNN architecture for object class segmentation is presented. We investigate several ways to train such a network. We evaluate our models on the challenging NYU Depth v2 dataset for object class segmentation and obtain competitive results. Copyright © 2017 Elsevier Ltd. All rights reserved.

Predicting non-linear dynamics by stable local learning in a recurrent spiking neural network.

PubMed

Gilra, Aditya; Gerstner, Wulfram

2017-11-27

The brain needs to predict how the body reacts to motor commands, but how a network of spiking neurons can learn non-linear body dynamics using local, online and stable learning rules is unclear. Here, we present a supervised learning scheme for the feedforward and recurrent connections in a network of heterogeneous spiking neurons. The error in the output is fed back through fixed random connections with a negative gain, causing the network to follow the desired dynamics. The rule for Feedback-based Online Local Learning Of Weights (FOLLOW) is local in the sense that weight changes depend on the presynaptic activity and the error signal projected onto the postsynaptic neuron. We provide examples of learning linear, non-linear and chaotic dynamics, as well as the dynamics of a two-link arm. Under reasonable approximations, we show, using the Lyapunov method, that FOLLOW learning is uniformly stable, with the error going to zero asymptotically.

Predicting non-linear dynamics by stable local learning in a recurrent spiking neural network

PubMed Central

Gerstner, Wulfram

2017-01-01

The brain needs to predict how the body reacts to motor commands, but how a network of spiking neurons can learn non-linear body dynamics using local, online and stable learning rules is unclear. Here, we present a supervised learning scheme for the feedforward and recurrent connections in a network of heterogeneous spiking neurons. The error in the output is fed back through fixed random connections with a negative gain, causing the network to follow the desired dynamics. The rule for Feedback-based Online Local Learning Of Weights (FOLLOW) is local in the sense that weight changes depend on the presynaptic activity and the error signal projected onto the postsynaptic neuron. We provide examples of learning linear, non-linear and chaotic dynamics, as well as the dynamics of a two-link arm. Under reasonable approximations, we show, using the Lyapunov method, that FOLLOW learning is uniformly stable, with the error going to zero asymptotically. PMID:29173280

Semantic disturbance in schizophrenia and its relationship to the cognitive neuroscience of attention

PubMed Central

Nestor, P.G.; Han, S.D.; Niznikiewicz, M.; Salisbury, D.; Spencer, K.; Shenton, M.E.; McCarley, R.W.

2010-01-01

We view schizophrenia as producing a failure of attentional modulation that leads to a breakdown in the selective enhancement or inhibition of semantic/lexical representations whose biological substrata are widely distributed across left (dominant) temporal and frontal lobes. Supporting behavioral evidence includes word recall studies that have pointed to a disturbance in connectivity (associative strength) but not network size (number of associates) in patients with schizophrenia. Paralleling these findings are recent neural network simulation studies of the abnormal connectivity effect in schizophrenia through ‘lesioning’ network connection weights while holding constant network size. Supporting evidence at the level of biology are in vitro studies examining N-methyl-d-aspartate (NMDA) receptor antagonists on recurrent inhibition; simulations in neural populations with realistically modeled biophysical properties show NMDA antagonists produce a schizophrenia-like disturbance in pattern association. We propose a similar failure of NMDA-mediated recurrent inhibition as a candidate biological substrate for attention and semantic anomalies of schizophrenia. PMID:11454433

A solution to neural field equations by a recurrent neural network method

NASA Astrophysics Data System (ADS)

Alharbi, Abir

2012-09-01

Neural field equations (NFE) are used to model the activity of neurons in the brain, it is introduced from a single neuron 'integrate-and-fire model' starting point. The neural continuum is spatially discretized for numerical studies, and the governing equations are modeled as a system of ordinary differential equations. In this article the recurrent neural network approach is used to solve this system of ODEs. This consists of a technique developed by combining the standard numerical method of finite-differences with the Hopfield neural network. The architecture of the net, energy function, updating equations, and algorithms are developed for the NFE model. A Hopfield Neural Network is then designed to minimize the energy function modeling the NFE. Results obtained from the Hopfield-finite-differences net show excellent performance in terms of accuracy and speed. The parallelism nature of the Hopfield approaches may make them easier to implement on fast parallel computers and give them the speed advantage over the traditional methods.

Association of Time between Surgery and Adjuvant Therapy with Survival in Oral Cavity Cancer.

PubMed

Chen, Michelle M; Harris, Jeremy P; Orosco, Ryan K; Sirjani, Davud; Hara, Wendy; Divi, Vasu

2018-06-01

Objective The National Cancer Center Network recommends starting radiation therapy within 6 weeks after surgery for oral cavity squamous cell carcinoma (OCSCC), but there is limited evidence of the importance of the total time from surgery to completion of radiation therapy (package time). We set out to determine if there was an association between package time and survival in OCSCC and to evaluate the impact of treatment location on outcomes. Study Design Retrospective cohort study. Setting Tertiary academic medical center. Subjects and Methods We reviewed the records of patients with OCSCC who completed postoperative radiation therapy at an academic medical center from 2008 to 2016. The primary endpoints were overall survival and recurrence-free survival. Statistical analysis included χ 2 tests and Cox proportional hazards regressions. Results We identified 132 patients with an average package time of 12.6 weeks. On multivariate analysis, package time >11 weeks was independently associated with decreased overall survival (hazard ratio, 6.68; 95% CI, 1.42-31.44) and recurrence-free survival (hazard ratio, 2.94; 95% CI, 1.20-7.18). Patients who received radiation therapy at outside facilities were more likely to have treatment delays (90.2% vs 62.9%, P = .001). Conclusions Prolonged package times are associated with decreased overall and recurrence-free survival among patients with OCSCC. Patients who received radiation therapy at outside facilities are more likely to have prolonged package times.

Contribution of artificial intelligence to the knowledge of prognostic factors in Hodgkin's lymphoma.

PubMed

Buciński, Adam; Marszałł, Michał Piotr; Krysiński, Jerzy; Lemieszek, Andrzej; Załuski, Jerzy

2010-07-01

Hodgkin's lymphoma is one of the most curable malignancies and most patients achieve a lasting complete remission. In this study, artificial neural network (ANN) analysis was shown to provide significant factors with regard to 5-year recurrence after lymphoma treatment. Data from 114 patients treated for Hodgkin's disease were available for evaluation and comparison. A total of 31 variables were subjected to ANN analysis. The ANN approach as an advanced multivariate data processing method was shown to provide objective prognostic data. Some of these prognostic factors are consistent or even identical to the factors evaluated earlier by other statistical methods.

Risk factors for and causes and treatment of recurrence of inferior vena cava type of Budd-Chiari syndrome after stenting in China: A retrospective analysis of a large cohort.

PubMed

Li, Wen-Dong; Yu, Hui-Ying; Qian, Ai-Min; Rong, Jian-Jie; Zhang, Ye-Qing; Li, Xiao-Qiang

2017-03-01

To explore the risk factors for recurrence of inferior vena cava (IVC)-type Budd-Chiari syndrome (BCS) after stenting and evaluate the feasibility and primary outcomes of endovascular therapies for recurrent BCS. A retrospective analysis of 219 patients was performed to identify risk factors for recurrence. The images of the recurrent patients during follow-up duration and interventional surgery were also reviewed to find the possible reasons of recurrence. The outcome of endovascular therapies for recurrent BCS was evaluated by Kaplan-Meier analysis. Among the 219 patients, 172 patients with primary IVC-type BCS underwent stenting and 28 patients experienced recurrence. Multivariate analysis identified age, Child-Pugh score, MELD and total bilirubin as independent recurrent indicators. Possible causes of recurrence include thrombosis in the stent, re-obstruction in or above the stent, and stent-related hepatic vein obstruction. Twenty-five patients with recurrent BCS underwent endovascular therapies with a few complications and achieved a high level of short- and mid-term patency. Age, total bilirubin and severity of liver function are the main risk factors for BCS recurrence. These risks might contribute to thrombosis or subsequent fibrous obstruction. Endovascular therapies are effective and safe management options that yield positive outcomes for recurrent BCS. • Risk factors for recurrent Budd-Chiari syndrome were identified by multivariate analysis. • Causes of recurrent Budd-Chiari syndrome were investigated by assessing radiological images. • There is a correlation between risk factors and causes of recurrence. • Endovascular therapies for recurrent Budd-Chiari syndrome are effective and safe.

Trainable Gene Regulation Networks with Applications to Drosophila Pattern Formation

NASA Technical Reports Server (NTRS)

Mjolsness, Eric

2000-01-01

This chapter will very briefly introduce and review some computational experiments in using trainable gene regulation network models to simulate and understand selected episodes in the development of the fruit fly, Drosophila melanogaster. For details the reader is referred to the papers introduced below. It will then introduce a new gene regulation network model which can describe promoter-level substructure in gene regulation. As described in chapter 2, gene regulation may be thought of as a combination of cis-acting regulation by the extended promoter of a gene (including all regulatory sequences) by way of the transcription complex, and of trans-acting regulation by the transcription factor products of other genes. If we simplify the cis-action by using a phenomenological model which can be tuned to data, such as a unit or other small portion of an artificial neural network, then the full transacting interaction between multiple genes during development can be modelled as a larger network which can again be tuned or trained to data. The larger network will in general need to have recurrent (feedback) connections since at least some real gene regulation networks do. This is the basic modeling approach taken, which describes how a set of recurrent neural networks can be used as a modeling language for multiple developmental processes including gene regulation within a single cell, cell-cell communication, and cell division. Such network models have been called "gene circuits", "gene regulation networks", or "genetic regulatory networks", sometimes without distinguishing the models from the actual modeled systems.

Developing robust recurrence plot analysis techniques for investigating infant respiratory patterns.

PubMed

Terrill, Philip I; Wilson, Stephen; Suresh, Sadasivam; Cooper, David M

2007-01-01

Recurrence plot analysis is a useful non-linear analysis tool. There are still no well formalised procedures for carrying out this analysis on measured physiological data, and systemising analysis is often difficult. In this paper, the recurrence based embedding is compared to radius based embedding by studying a logistic attractor and measured breathing data collected from sleeping human infants. Recurrence based embedding appears to be a more robust method of carrying out a recurrence analysis when attractor size is likely to be different between datasets. In the infant breathing data, the radius measure calculated at a fixed recurrence, scaled by average respiratory period, allows the accurate discrimination of active sleep from quiet sleep states (AUC=0.975, Sn=098, Sp=0.94).

Development and validation of automatic tools for interactive recurrence analysis in radiation therapy: optimization of treatment algorithms for locally advanced pancreatic cancer.

PubMed

Kessel, Kerstin A; Habermehl, Daniel; Jäger, Andreas; Floca, Ralf O; Zhang, Lanlan; Bendl, Rolf; Debus, Jürgen; Combs, Stephanie E

2013-06-07

In radiation oncology recurrence analysis is an important part in the evaluation process and clinical quality assurance of treatment concepts. With the example of 9 patients with locally advanced pancreatic cancer we developed and validated interactive analysis tools to support the evaluation workflow. After an automatic registration of the radiation planning CTs with the follow-up images, the recurrence volumes are segmented manually. Based on these volumes the DVH (dose volume histogram) statistic is calculated, followed by the determination of the dose applied to the region of recurrence and the distance between the boost and recurrence volume. We calculated the percentage of the recurrence volume within the 80%-isodose volume and compared it to the location of the recurrence within the boost volume, boost + 1 cm, boost + 1.5 cm and boost + 2 cm volumes. Recurrence analysis of 9 patients demonstrated that all recurrences except one occurred within the defined GTV/boost volume; one recurrence developed beyond the field border/outfield. With the defined distance volumes in relation to the recurrences, we could show that 7 recurrent lesions were within the 2 cm radius of the primary tumor. Two large recurrences extended beyond the 2 cm, however, this might be due to very rapid growth and/or late detection of the tumor progression. The main goal of using automatic analysis tools is to reduce time and effort conducting clinical analyses. We showed a first approach and use of a semi-automated workflow for recurrence analysis, which will be continuously optimized. In conclusion, despite the limitations of the automatic calculations we contributed to in-house optimization of subsequent study concepts based on an improved and validated target volume definition.

Effect of dilution in asymmetric recurrent neural networks.

PubMed

Folli, Viola; Gosti, Giorgio; Leonetti, Marco; Ruocco, Giancarlo

2018-04-16

We study with numerical simulation the possible limit behaviors of synchronous discrete-time deterministic recurrent neural networks composed of N binary neurons as a function of a network's level of dilution and asymmetry. The network dilution measures the fraction of neuron couples that are connected, and the network asymmetry measures to what extent the underlying connectivity matrix is asymmetric. For each given neural network, we study the dynamical evolution of all the different initial conditions, thus characterizing the full dynamical landscape without imposing any learning rule. Because of the deterministic dynamics, each trajectory converges to an attractor, that can be either a fixed point or a limit cycle. These attractors form the set of all the possible limit behaviors of the neural network. For each network we then determine the convergence times, the limit cycles' length, the number of attractors, and the sizes of the attractors' basin. We show that there are two network structures that maximize the number of possible limit behaviors. The first optimal network structure is fully-connected and symmetric. On the contrary, the second optimal network structure is highly sparse and asymmetric. The latter optimal is similar to what observed in different biological neuronal circuits. These observations lead us to hypothesize that independently from any given learning model, an efficient and effective biologic network that stores a number of limit behaviors close to its maximum capacity tends to develop a connectivity structure similar to one of the optimal networks we found. Copyright © 2018 The Author(s). Published by Elsevier Ltd.. All rights reserved.

A model of human motor sequence learning explains facilitation and interference effects based on spike-timing dependent plasticity.

PubMed

Wang, Quan; Rothkopf, Constantin A; Triesch, Jochen

2017-08-01

The ability to learn sequential behaviors is a fundamental property of our brains. Yet a long stream of studies including recent experiments investigating motor sequence learning in adult human subjects have produced a number of puzzling and seemingly contradictory results. In particular, when subjects have to learn multiple action sequences, learning is sometimes impaired by proactive and retroactive interference effects. In other situations, however, learning is accelerated as reflected in facilitation and transfer effects. At present it is unclear what the underlying neural mechanism are that give rise to these diverse findings. Here we show that a recently developed recurrent neural network model readily reproduces this diverse set of findings. The self-organizing recurrent neural network (SORN) model is a network of recurrently connected threshold units that combines a simplified form of spike-timing dependent plasticity (STDP) with homeostatic plasticity mechanisms ensuring network stability, namely intrinsic plasticity (IP) and synaptic normalization (SN). When trained on sequence learning tasks modeled after recent experiments we find that it reproduces the full range of interference, facilitation, and transfer effects. We show how these effects are rooted in the network's changing internal representation of the different sequences across learning and how they depend on an interaction of training schedule and task similarity. Furthermore, since learning in the model is based on fundamental neuronal plasticity mechanisms, the model reveals how these plasticity mechanisms are ultimately responsible for the network's sequence learning abilities. In particular, we find that all three plasticity mechanisms are essential for the network to learn effective internal models of the different training sequences. This ability to form effective internal models is also the basis for the observed interference and facilitation effects. This suggests that STDP, IP, and SN may be the driving forces behind our ability to learn complex action sequences.

Memory Retrieval Time and Memory Capacity of the CA3 Network: Role of Gamma Frequency Oscillations

ERIC Educational Resources Information Center

de Almeida, Licurgo; Idiart, Marco; Lisman, John E.

2007-01-01

The existence of recurrent synaptic connections in CA3 led to the hypothesis that CA3 is an autoassociative network similar to the Hopfield networks studied by theorists. CA3 undergoes gamma frequency periodic inhibition that prevents a persistent attractor state. This argues against the analogy to Hopfield nets, in which an attractor state can be…

Management of recurrent and persistent metastatic lymph nodes in well-differentiated thyroid cancer: a multifactorial decision-making guide for the Thyroid Cancer Care Collaborative.

PubMed

Urken, Mark L; Milas, Mira; Randolph, Gregory W; Tufano, Ralph; Bergman, Donald; Bernet, Victor; Brett, Elise M; Brierley, James D; Cobin, Rhoda; Doherty, Gerard; Klopper, Joshua; Lee, Stephanie; Machac, Josef; Mechanick, Jeffrey I; Orloff, Lisa A; Ross, Douglas; Smallridge, Robert C; Terris, David J; Clain, Jason B; Tuttle, Michael

2015-04-01

Well-differentiated thyroid cancer (WDTC) recurs in up to 30% of patients. Guidelines from the American Thyroid Association (ATA) and the National Comprehensive Cancer Network (NCCN) provide valuable parameters for the management of recurrent disease, but fail to guide the clinician as to the multitude of factors that should be taken into account. The Thyroid Cancer Care Collaborative (TCCC) is a web-based repository of a patient's clinical information. Ten clinical decision-making modules (CDMMs) process this information and display individualized treatment recommendations. We conducted a review of the literature and analysis of the management of patients with recurrent/persistent WDTC. Surgery remains the most common treatment in recurrent/persistent WDTC and can be performed with limited morbidity in experienced hands. However, careful observation may be the recommended course in select patients. Reoperation yields biochemical remission rates between 21% and 66%. There is a reported 1.2% incidence of permanent unexpected nerve paralysis and a 3.5% incidence of permanent hypoparathyroidism. External beam radiotherapy and percutaneous ethanol ablation have been reported as therapeutic alternatives. Radioactive iodine as a primary therapy has been reported previously for metastatic lymph nodes, but is currently advocated by the ATA as an adjuvant to surgery. The management of recurrent lymph nodes is a multifactorial decision and is best determined by a multidisciplinary team. The CDMMs allow for easy adoption of contemporary knowledge, making this information accessible to both patient and clinician. © 2014 Wiley Periodicals, Inc.

Microarray analysis of potential genes in the pathogenesis of recurrent oral ulcer.

PubMed

Han, Jingying; He, Zhiwei; Li, Kun; Hou, Lu

2015-01-01

Recurrent oral ulcer seriously threatens patients' daily life and health. This study investigated potential genes and pathways that participate in the pathogenesis of recurrent oral ulcer by high throughput bioinformatic analysis. RT-PCR and Western blot were applied to further verify screened interleukins effect. Recurrent oral ulcer related genes were collected from websites and papers, and further found out from Human Genome 280 6.0 microarray data. Each pathway of recurrent oral ulcer related genes were got through chip hybridization. RT-PCR was applied to test four recurrent oral ulcer related genes to verify the microarray data. Data transformation, scatter plot, clustering analysis, and expression pattern analysis were used to analyze recurrent oral ulcer related gene expression changes. Recurrent oral ulcer gene microarray was successfully established. Microarray showed that 551 genes involved in recurrent oral ulcer activity and 196 genes were recurrent oral ulcer related genes. Of them, 76 genes up-regulated, 62 genes down-regulated, and 58 genes up-/down-regulated. Total expression level up-regulated 752 times (60%) and down-regulated 485 times (40%). IL-2 plays an important role in the occurrence, development and recurrence of recurrent oral ulcer on the mRNA and protein levels. Gene microarray can be used to analyze potential genes and pathways in recurrent oral ulcer. IL-2 may be involved in the pathogenesis of recurrent oral ulcer.

Recurrent neural network based virtual detection line

NASA Astrophysics Data System (ADS)

Kadikis, Roberts

2018-04-01

The paper proposes an efficient method for detection of moving objects in the video. The objects are detected when they cross a virtual detection line. Only the pixels of the detection line are processed, which makes the method computationally efficient. A Recurrent Neural Network processes these pixels. The machine learning approach allows one to train a model that works in different and changing outdoor conditions. Also, the same network can be trained for various detection tasks, which is demonstrated by the tests on vehicle and people counting. In addition, the paper proposes a method for semi-automatic acquisition of labeled training data. The labeling method is used to create training and testing datasets, which in turn are used to train and evaluate the accuracy and efficiency of the detection method. The method shows similar accuracy as the alternative efficient methods but provides greater adaptability and usability for different tasks.

Dual-phase evolution in complex adaptive systems

PubMed Central

Paperin, Greg; Green, David G.; Sadedin, Suzanne

2011-01-01

Understanding the origins of complexity is a key challenge in many sciences. Although networks are known to underlie most systems, showing how they contribute to well-known phenomena remains an issue. Here, we show that recurrent phase transitions in network connectivity underlie emergent phenomena in many systems. We identify properties that are typical of systems in different connectivity phases, as well as characteristics commonly associated with the phase transitions. We synthesize these common features into a common framework, which we term dual-phase evolution (DPE). Using this framework, we review the literature from several disciplines to show that recurrent connectivity phase transitions underlie the complex properties of many biological, physical and human systems. We argue that the DPE framework helps to explain many complex phenomena, including perpetual novelty, modularity, scale-free networks and criticality. Our review concludes with a discussion of the way DPE relates to other frameworks, in particular, self-organized criticality and the adaptive cycle. PMID:21247947

Dual-phase evolution in complex adaptive systems.

PubMed

Paperin, Greg; Green, David G; Sadedin, Suzanne

2011-05-06

Understanding the origins of complexity is a key challenge in many sciences. Although networks are known to underlie most systems, showing how they contribute to well-known phenomena remains an issue. Here, we show that recurrent phase transitions in network connectivity underlie emergent phenomena in many systems. We identify properties that are typical of systems in different connectivity phases, as well as characteristics commonly associated with the phase transitions. We synthesize these common features into a common framework, which we term dual-phase evolution (DPE). Using this framework, we review the literature from several disciplines to show that recurrent connectivity phase transitions underlie the complex properties of many biological, physical and human systems. We argue that the DPE framework helps to explain many complex phenomena, including perpetual novelty, modularity, scale-free networks and criticality. Our review concludes with a discussion of the way DPE relates to other frameworks, in particular, self-organized criticality and the adaptive cycle.

Multiscale recurrence quantification analysis of order recurrence plots

NASA Astrophysics Data System (ADS)

Xu, Mengjia; Shang, Pengjian; Lin, Aijing

2017-03-01

In this paper, we propose a new method of multiscale recurrence quantification analysis (MSRQA) to analyze the structure of order recurrence plots. The MSRQA is based on order patterns over a range of time scales. Compared with conventional recurrence quantification analysis (RQA), the MSRQA can show richer and more recognizable information on the local characteristics of diverse systems which successfully describes their recurrence properties. Both synthetic series and stock market indexes exhibit their properties of recurrence at large time scales that quite differ from those at a single time scale. Some systems present more accurate recurrence patterns under large time scales. It demonstrates that the new approach is effective for distinguishing three similar stock market systems and showing some inherent differences.

Natural Language Video Description using Deep Recurrent Neural Networks

DTIC Science & Technology

2015-11-23

records who says what, but lacks tim- ing information. Movie scripts typically include names of all characters and most movies loosely follow the...and Jürgen Schmidhuber. A novel approach to on-line handwriting recognition based on bidirectional long short-term memory networks. In Proc. 9th Int

Recurrence quantification analysis of electrically evoked surface EMG signal.

PubMed

Liu, Chunling; Wang, Xu

2005-01-01

Recurrence Plot is a quite useful tool used in time-series analysis, in particular for measuring unstable periodic orbits embedded in a chaotic dynamical system. This paper introduced the structures of the Recurrence Plot and the ways of the plot coming into being. Then the way of the quantification of the Recurrence Plot is defined. In this paper, one of the possible applications of Recurrence Quantification Analysis (RQA) strategy to the analysis of electrical stimulation evoked surface EMG. The result shows the percent determination is increased along with stimulation intensity.

Learning to Generate Sequences with Combination of Hebbian and Non-hebbian Plasticity in Recurrent Spiking Neural Networks

PubMed Central

Panda, Priyadarshini; Roy, Kaushik

2017-01-01

Synaptic Plasticity, the foundation for learning and memory formation in the human brain, manifests in various forms. Here, we combine the standard spike timing correlation based Hebbian plasticity with a non-Hebbian synaptic decay mechanism for training a recurrent spiking neural model to generate sequences. We show that inclusion of the adaptive decay of synaptic weights with standard STDP helps learn stable contextual dependencies between temporal sequences, while reducing the strong attractor states that emerge in recurrent models due to feedback loops. Furthermore, we show that the combined learning scheme suppresses the chaotic activity in the recurrent model substantially, thereby enhancing its' ability to generate sequences consistently even in the presence of perturbations. PMID:29311774

Temporal neural networks and transient analysis of complex engineering systems

NASA Astrophysics Data System (ADS)

Uluyol, Onder

A theory is introduced for a multi-layered Local Output Gamma Feedback (LOGF) neural network within the paradigm of Locally-Recurrent Globally-Feedforward neural networks. It is developed for the identification, prediction, and control tasks of spatio-temporal systems and allows for the presentation of different time scales through incorporation of a gamma memory. It is initially applied to the tasks of sunspot and Mackey-Glass series prediction as benchmarks, then it is extended to the task of power level control of a nuclear reactor at different fuel cycle conditions. The developed LOGF neuron model can also be viewed as a Transformed Input and State (TIS) Gamma memory for neural network architectures for temporal processing. The novel LOGF neuron model extends the static neuron model by incorporating into it a short-term memory structure in the form of a digital gamma filter. A feedforward neural network made up of LOGF neurons can thus be used to model dynamic systems. A learning algorithm based upon the Backpropagation-Through-Time (BTT) approach is derived. It is applicable for training a general L-layer LOGF neural network. The spatial and temporal weights and parameters of the network are iteratively optimized for a given problem using the derived learning algorithm.

Cross-evidence for hypnotic susceptibility through nonlinear measures on EEGs of non-hypnotized subjects

NASA Astrophysics Data System (ADS)

Chiarucci, Riccardo; Madeo, Dario; Loffredo, Maria I.; Castellani, Eleonora; Santarcangelo, Enrica L.; Mocenni, Chiara

2014-07-01

Assessment of hypnotic susceptibility is usually obtained through the application of psychological instruments. A satisfying classification obtained through quantitative measures is still missing, although it would be very useful for both diagnostic and clinical purposes. Aiming at investigating the relationship between the cortical brain activity and the hypnotic susceptibility level, we propose the combined use of two methodologies - Recurrence Quantification Analysis and Detrended Fluctuation Analysis - both inherited from nonlinear dynamics. Indicators obtained through the application of these techniques to EEG signals of individuals in their ordinary state of consciousness allowed us to obtain a clear discrimination between subjects with high and low susceptibility to hypnosis. Finally a neural network approach was used to perform classification analysis.

Continuous Timescale Long-Short Term Memory Neural Network for Human Intent Understanding

PubMed Central

Yu, Zhibin; Moirangthem, Dennis S.; Lee, Minho

2017-01-01

Understanding of human intention by observing a series of human actions has been a challenging task. In order to do so, we need to analyze longer sequences of human actions related with intentions and extract the context from the dynamic features. The multiple timescales recurrent neural network (MTRNN) model, which is believed to be a kind of solution, is a useful tool for recording and regenerating a continuous signal for dynamic tasks. However, the conventional MTRNN suffers from the vanishing gradient problem which renders it impossible to be used for longer sequence understanding. To address this problem, we propose a new model named Continuous Timescale Long-Short Term Memory (CTLSTM) in which we inherit the multiple timescales concept into the Long-Short Term Memory (LSTM) recurrent neural network (RNN) that addresses the vanishing gradient problem. We design an additional recurrent connection in the LSTM cell outputs to produce a time-delay in order to capture the slow context. Our experiments show that the proposed model exhibits better context modeling ability and captures the dynamic features on multiple large dataset classification tasks. The results illustrate that the multiple timescales concept enhances the ability of our model to handle longer sequences related with human intentions and hence proving to be more suitable for complex tasks, such as intention recognition. PMID:28878646

Structured Semantic Knowledge Can Emerge Automatically from Predicting Word Sequences in Child-Directed Speech

PubMed Central

Huebner, Philip A.; Willits, Jon A.

2018-01-01

Previous research has suggested that distributional learning mechanisms may contribute to the acquisition of semantic knowledge. However, distributional learning mechanisms, statistical learning, and contemporary “deep learning” approaches have been criticized for being incapable of learning the kind of abstract and structured knowledge that many think is required for acquisition of semantic knowledge. In this paper, we show that recurrent neural networks, trained on noisy naturalistic speech to children, do in fact learn what appears to be abstract and structured knowledge. We trained two types of recurrent neural networks (Simple Recurrent Network, and Long Short-Term Memory) to predict word sequences in a 5-million-word corpus of speech directed to children ages 0–3 years old, and assessed what semantic knowledge they acquired. We found that learned internal representations are encoding various abstract grammatical and semantic features that are useful for predicting word sequences. Assessing the organization of semantic knowledge in terms of the similarity structure, we found evidence of emergent categorical and hierarchical structure in both models. We found that the Long Short-term Memory (LSTM) and SRN are both learning very similar kinds of representations, but the LSTM achieved higher levels of performance on a quantitative evaluation. We also trained a non-recurrent neural network, Skip-gram, on the same input to compare our results to the state-of-the-art in machine learning. We found that Skip-gram achieves relatively similar performance to the LSTM, but is representing words more in terms of thematic compared to taxonomic relations, and we provide reasons why this might be the case. Our findings show that a learning system that derives abstract, distributed representations for the purpose of predicting sequential dependencies in naturalistic language may provide insight into emergence of many properties of the developing semantic system. PMID:29520243

Real time unsupervised learning of visual stimuli in neuromorphic VLSI systems

NASA Astrophysics Data System (ADS)

Giulioni, Massimiliano; Corradi, Federico; Dante, Vittorio; Del Giudice, Paolo

2015-10-01

Neuromorphic chips embody computational principles operating in the nervous system, into microelectronic devices. In this domain it is important to identify computational primitives that theory and experiments suggest as generic and reusable cognitive elements. One such element is provided by attractor dynamics in recurrent networks. Point attractors are equilibrium states of the dynamics (up to fluctuations), determined by the synaptic structure of the network; a ‘basin’ of attraction comprises all initial states leading to a given attractor upon relaxation, hence making attractor dynamics suitable to implement robust associative memory. The initial network state is dictated by the stimulus, and relaxation to the attractor state implements the retrieval of the corresponding memorized prototypical pattern. In a previous work we demonstrated that a neuromorphic recurrent network of spiking neurons and suitably chosen, fixed synapses supports attractor dynamics. Here we focus on learning: activating on-chip synaptic plasticity and using a theory-driven strategy for choosing network parameters, we show that autonomous learning, following repeated presentation of simple visual stimuli, shapes a synaptic connectivity supporting stimulus-selective attractors. Associative memory develops on chip as the result of the coupled stimulus-driven neural activity and ensuing synaptic dynamics, with no artificial separation between learning and retrieval phases.

Real time unsupervised learning of visual stimuli in neuromorphic VLSI systems.

PubMed

Giulioni, Massimiliano; Corradi, Federico; Dante, Vittorio; del Giudice, Paolo

2015-10-14

Neuromorphic chips embody computational principles operating in the nervous system, into microelectronic devices. In this domain it is important to identify computational primitives that theory and experiments suggest as generic and reusable cognitive elements. One such element is provided by attractor dynamics in recurrent networks. Point attractors are equilibrium states of the dynamics (up to fluctuations), determined by the synaptic structure of the network; a 'basin' of attraction comprises all initial states leading to a given attractor upon relaxation, hence making attractor dynamics suitable to implement robust associative memory. The initial network state is dictated by the stimulus, and relaxation to the attractor state implements the retrieval of the corresponding memorized prototypical pattern. In a previous work we demonstrated that a neuromorphic recurrent network of spiking neurons and suitably chosen, fixed synapses supports attractor dynamics. Here we focus on learning: activating on-chip synaptic plasticity and using a theory-driven strategy for choosing network parameters, we show that autonomous learning, following repeated presentation of simple visual stimuli, shapes a synaptic connectivity supporting stimulus-selective attractors. Associative memory develops on chip as the result of the coupled stimulus-driven neural activity and ensuing synaptic dynamics, with no artificial separation between learning and retrieval phases.

A Recurrent Network Model of Somatosensory Parametric Working Memory in the Prefrontal Cortex

PubMed Central

Miller, Paul; Brody, Carlos D; Romo, Ranulfo; Wang, Xiao-Jing

2015-01-01

A parametric working memory network stores the information of an analog stimulus in the form of persistent neural activity that is monotonically tuned to the stimulus. The family of persistent firing patterns with a continuous range of firing rates must all be realizable under exactly the same external conditions (during the delay when the transient stimulus is withdrawn). How this can be accomplished by neural mechanisms remains an unresolved question. Here we present a recurrent cortical network model of irregularly spiking neurons that was designed to simulate a somatosensory working memory experiment with behaving monkeys. Our model reproduces the observed positively and negatively monotonic persistent activity, and heterogeneous tuning curves of memory activity. We show that fine-tuning mathematically corresponds to a precise alignment of cusps in the bifurcation diagram of the network. Moreover, we show that the fine-tuned network can integrate stimulus inputs over several seconds. Assuming that such time integration occurs in neural populations downstream from a tonically persistent neural population, our model is able to account for the slow ramping-up and ramping-down behaviors of neurons observed in prefrontal cortex. PMID:14576212

Classification of conductance traces with recurrent neural networks

NASA Astrophysics Data System (ADS)

Lauritzen, Kasper P.; Magyarkuti, András; Balogh, Zoltán; Halbritter, András; Solomon, Gemma C.

2018-02-01

We present a new automated method for structural classification of the traces obtained in break junction experiments. Using recurrent neural networks trained on the traces of minimal cross-sectional area in molecular dynamics simulations, we successfully separate the traces into two classes: point contact or nanowire. This is done without any assumptions about the expected features of each class. The trained neural network is applied to experimental break junction conductance traces, and it separates the classes as well as the previously used experimental methods. The effect of using partial conductance traces is explored, and we show that the method performs equally well using full or partial traces (as long as the trace just prior to breaking is included). When only the initial part of the trace is included, the results are still better than random chance. Finally, we show that the neural network classification method can be used to classify experimental conductance traces without using simulated results for training, but instead training the network on a few representative experimental traces. This offers a tool to recognize some characteristic motifs of the traces, which can be hard to find by simple data selection algorithms.

Implicity Defined Neural Networks for Sequence Labeling

DTIC Science & Technology

2017-02-13

popularity of the Long Short - Term Memory (LSTM) (Hochreiter and Schmidhuber, 1997) and variants such as the Gated Recurrent Unit (GRU) (Cho et al., 2014...bidirectional lstm and other neural network architectures. Neural Net- works 18(5):602–610. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short - term ...hid- den states of the network to coupled together, allowing potential improvement on problems with complex, long -distance dependencies. Initial

Spatiotemporal Dynamics and Reliable Computations in Recurrent Spiking Neural Networks

NASA Astrophysics Data System (ADS)

Pyle, Ryan; Rosenbaum, Robert

2017-01-01

Randomly connected networks of excitatory and inhibitory spiking neurons provide a parsimonious model of neural variability, but are notoriously unreliable for performing computations. We show that this difficulty is overcome by incorporating the well-documented dependence of connection probability on distance. Spatially extended spiking networks exhibit symmetry-breaking bifurcations and generate spatiotemporal patterns that can be trained to perform dynamical computations under a reservoir computing framework.

Recurrent neural network-based modeling of gene regulatory network using elephant swarm water search algorithm.

PubMed

Mandal, Sudip; Saha, Goutam; Pal, Rajat Kumar

2017-08-01

Correct inference of genetic regulations inside a cell from the biological database like time series microarray data is one of the greatest challenges in post genomic era for biologists and researchers. Recurrent Neural Network (RNN) is one of the most popular and simple approach to model the dynamics as well as to infer correct dependencies among genes. Inspired by the behavior of social elephants, we propose a new metaheuristic namely Elephant Swarm Water Search Algorithm (ESWSA) to infer Gene Regulatory Network (GRN). This algorithm is mainly based on the water search strategy of intelligent and social elephants during drought, utilizing the different types of communication techniques. Initially, the algorithm is tested against benchmark small and medium scale artificial genetic networks without and with presence of different noise levels and the efficiency was observed in term of parametric error, minimum fitness value, execution time, accuracy of prediction of true regulation, etc. Next, the proposed algorithm is tested against the real time gene expression data of Escherichia Coli SOS Network and results were also compared with others state of the art optimization methods. The experimental results suggest that ESWSA is very efficient for GRN inference problem and performs better than other methods in many ways.

An LMI approach to design H(infinity) controllers for discrete-time nonlinear systems based on unified models.

PubMed

Liu, Meiqin; Zhang, Senlin

2008-10-01

A unified neural network model termed standard neural network model (SNNM) is advanced. Based on the robust L(2) gain (i.e. robust H(infinity) performance) analysis of the SNNM with external disturbances, a state-feedback control law is designed for the SNNM to stabilize the closed-loop system and eliminate the effect of external disturbances. The control design constraints are shown to be a set of linear matrix inequalities (LMIs) which can be easily solved by various convex optimization algorithms (e.g. interior-point algorithms) to determine the control law. Most discrete-time recurrent neural network (RNNs) and discrete-time nonlinear systems modelled by neural networks or Takagi and Sugeno (T-S) fuzzy models can be transformed into the SNNMs to be robust H(infinity) performance analyzed or robust H(infinity) controller synthesized in a unified SNNM's framework. Finally, some examples are presented to illustrate the wide application of the SNNMs to the nonlinear systems, and the proposed approach is compared with related methods reported in the literature.

Recurrence plots and recurrence quantification analysis of human motion data

NASA Astrophysics Data System (ADS)

Josiński, Henryk; Michalczuk, Agnieszka; Świtoński, Adam; Szczesna, Agnieszka; Wojciechowski, Konrad

2016-06-01

The authors present exemplary application of recurrence plots, cross recurrence plots and recurrence quantification analysis for the purpose of exploration of experimental time series describing selected aspects of human motion. Time series were extracted from treadmill gait sequences which were recorded in the Human Motion Laboratory (HML) of the Polish-Japanese Academy of Information Technology in Bytom, Poland by means of the Vicon system. Analysis was focused on the time series representing movements of hip, knee, ankle and wrist joints in the sagittal plane.

Comparison of the dynamics of neural interactions between current-based and conductance-based integrate-and-fire recurrent networks

PubMed Central

Cavallari, Stefano; Panzeri, Stefano; Mazzoni, Alberto

2014-01-01

Models of networks of Leaky Integrate-and-Fire (LIF) neurons are a widely used tool for theoretical investigations of brain function. These models have been used both with current- and conductance-based synapses. However, the differences in the dynamics expressed by these two approaches have been so far mainly studied at the single neuron level. To investigate how these synaptic models affect network activity, we compared the single neuron and neural population dynamics of conductance-based networks (COBNs) and current-based networks (CUBNs) of LIF neurons. These networks were endowed with sparse excitatory and inhibitory recurrent connections, and were tested in conditions including both low- and high-conductance states. We developed a novel procedure to obtain comparable networks by properly tuning the synaptic parameters not shared by the models. The so defined comparable networks displayed an excellent and robust match of first order statistics (average single neuron firing rates and average frequency spectrum of network activity). However, these comparable networks showed profound differences in the second order statistics of neural population interactions and in the modulation of these properties by external inputs. The correlation between inhibitory and excitatory synaptic currents and the cross-neuron correlation between synaptic inputs, membrane potentials and spike trains were stronger and more stimulus-modulated in the COBN. Because of these properties, the spike train correlation carried more information about the strength of the input in the COBN, although the firing rates were equally informative in both network models. Moreover, the network activity of COBN showed stronger synchronization in the gamma band, and spectral information about the input higher and spread over a broader range of frequencies. These results suggest that the second order statistics of network dynamics depend strongly on the choice of synaptic model. PMID:24634645

Comparison of the dynamics of neural interactions between current-based and conductance-based integrate-and-fire recurrent networks.

PubMed

Cavallari, Stefano; Panzeri, Stefano; Mazzoni, Alberto

2014-01-01

Models of networks of Leaky Integrate-and-Fire (LIF) neurons are a widely used tool for theoretical investigations of brain function. These models have been used both with current- and conductance-based synapses. However, the differences in the dynamics expressed by these two approaches have been so far mainly studied at the single neuron level. To investigate how these synaptic models affect network activity, we compared the single neuron and neural population dynamics of conductance-based networks (COBNs) and current-based networks (CUBNs) of LIF neurons. These networks were endowed with sparse excitatory and inhibitory recurrent connections, and were tested in conditions including both low- and high-conductance states. We developed a novel procedure to obtain comparable networks by properly tuning the synaptic parameters not shared by the models. The so defined comparable networks displayed an excellent and robust match of first order statistics (average single neuron firing rates and average frequency spectrum of network activity). However, these comparable networks showed profound differences in the second order statistics of neural population interactions and in the modulation of these properties by external inputs. The correlation between inhibitory and excitatory synaptic currents and the cross-neuron correlation between synaptic inputs, membrane potentials and spike trains were stronger and more stimulus-modulated in the COBN. Because of these properties, the spike train correlation carried more information about the strength of the input in the COBN, although the firing rates were equally informative in both network models. Moreover, the network activity of COBN showed stronger synchronization in the gamma band, and spectral information about the input higher and spread over a broader range of frequencies. These results suggest that the second order statistics of network dynamics depend strongly on the choice of synaptic model.

Network model of top-down influences on local gain and contextual interactions in visual cortex.

PubMed

Piëch, Valentin; Li, Wu; Reeke, George N; Gilbert, Charles D

2013-10-22

The visual system uses continuity as a cue for grouping oriented line segments that define object boundaries in complex visual scenes. Many studies support the idea that long-range intrinsic horizontal connections in early visual cortex contribute to this grouping. Top-down influences in primary visual cortex (V1) play an important role in the processes of contour integration and perceptual saliency, with contour-related responses being task dependent. This suggests an interaction between recurrent inputs to V1 and intrinsic connections within V1 that enables V1 neurons to respond differently under different conditions. We created a network model that simulates parametrically the control of local gain by hypothetical top-down modification of local recurrence. These local gain changes, as a consequence of network dynamics in our model, enable modulation of contextual interactions in a task-dependent manner. Our model displays contour-related facilitation of neuronal responses and differential foreground vs. background responses over the neuronal ensemble, accounting for the perceptual pop-out of salient contours. It quantitatively reproduces the results of single-unit recording experiments in V1, highlighting salient contours and replicating the time course of contextual influences. We show by means of phase-plane analysis that the model operates stably even in the presence of large inputs. Our model shows how a simple form of top-down modulation of the effective connectivity of intrinsic cortical connections among biophysically realistic neurons can account for some of the response changes seen in perceptual learning and task switching.

Generalized Recurrent Neural Network accommodating Dynamic Causal Modeling for functional MRI analysis.

PubMed

Wang, Yuan; Wang, Yao; Lui, Yvonne W

2018-05-18

Dynamic Causal Modeling (DCM) is an advanced biophysical model which explicitly describes the entire process from experimental stimuli to functional magnetic resonance imaging (fMRI) signals via neural activity and cerebral hemodynamics. To conduct a DCM study, one needs to represent the experimental stimuli as a compact vector-valued function of time, which is hard in complex tasks such as book reading and natural movie watching. Deep learning provides the state-of-the-art signal representation solution, encoding complex signals into compact dense vectors while preserving the essence of the original signals. There is growing interest in using Recurrent Neural Networks (RNNs), a major family of deep learning techniques, in fMRI modeling. However, the generic RNNs used in existing studies work as black boxes, making the interpretation of results in a neuroscience context difficult and obscure. In this paper, we propose a new biophysically interpretable RNN built on DCM, DCM-RNN. We generalize the vanilla RNN and show that DCM can be cast faithfully as a special form of the generalized RNN. DCM-RNN uses back propagation for parameter estimation. We believe DCM-RNN is a promising tool for neuroscience. It can fit seamlessly into classical DCM studies. We demonstrate face validity of DCM-RNN in two principal applications of DCM: causal brain architecture hypotheses testing and effective connectivity estimation. We also demonstrate construct validity of DCM-RNN in an attention-visual experiment. Moreover, DCM-RNN enables end-to-end training of DCM and representation learning deep neural networks, extending DCM studies to complex tasks. Copyright © 2018 Elsevier Inc. All rights reserved.

Timescales and Mechanisms of Sigh-Like Bursting and Spiking in Models of Rhythmic Respiratory Neurons.

PubMed

Wang, Yangyang; Rubin, Jonathan E

2017-12-01

Neural networks generate a variety of rhythmic activity patterns, often involving different timescales. One example arises in the respiratory network in the pre-Bötzinger complex of the mammalian brainstem, which can generate the eupneic rhythm associated with normal respiration as well as recurrent low-frequency, large-amplitude bursts associated with sighing. Two competing hypotheses have been proposed to explain sigh generation: the recruitment of a neuronal population distinct from the eupneic rhythm-generating subpopulation or the reconfiguration of activity within a single population. Here, we consider two recent computational models, one of which represents each of the hypotheses. We use methods of dynamical systems theory, such as fast-slow decomposition, averaging, and bifurcation analysis, to understand the multiple-timescale mechanisms underlying sigh generation in each model. In the course of our analysis, we discover that a third timescale is required to generate sighs in both models. Furthermore, we identify the similarities of the underlying mechanisms in the two models and the aspects in which they differ.

Toyota Prius HEV neurocontrol and diagnostics.

PubMed

Prokhorov, Danil V

2008-01-01

A neural network controller for improved fuel efficiency of the Toyota Prius hybrid electric vehicle is proposed. A new method to detect and mitigate a battery fault is also presented. The approach is based on recurrent neural networks and includes the extended Kalman filter. The proposed approach is quite general and applicable to other control systems.

Speaker-dependent Multipitch Tracking Using Deep Neural Networks

DTIC Science & Technology

2015-01-01

connections through time. Studies have shown that RNNs are good at modeling sequential data like handwriting [12] and speech [26]. We plan to explore RNNs in...Schmidhuber, and S. Fernández, “Unconstrained on-line handwriting recognition with recurrent neural networks,” in Proceedings of NIPS, 2008, pp. 577–584. [13

Assessing the Role of Inhibition in Stabilizing Neocortical Networks Requires Large-Scale Perturbation of the Inhibitory Population

PubMed Central

Mrsic-Flogel, Thomas D.

2017-01-01

Neurons within cortical microcircuits are interconnected with recurrent excitatory synaptic connections that are thought to amplify signals (Douglas and Martin, 2007), form selective subnetworks (Ko et al., 2011), and aid feature discrimination. Strong inhibition (Haider et al., 2013) counterbalances excitation, enabling sensory features to be sharpened and represented by sparse codes (Willmore et al., 2011). This balance between excitation and inhibition makes it difficult to assess the strength, or gain, of recurrent excitatory connections within cortical networks, which is key to understanding their operational regime and the computations that they perform. Networks that combine an unstable high-gain excitatory population with stabilizing inhibitory feedback are known as inhibition-stabilized networks (ISNs) (Tsodyks et al., 1997). Theoretical studies using reduced network models predict that ISNs produce paradoxical responses to perturbation, but experimental perturbations failed to find evidence for ISNs in cortex (Atallah et al., 2012). Here, we reexamined this question by investigating how cortical network models consisting of many neurons behave after perturbations and found that results obtained from reduced network models fail to predict responses to perturbations in more realistic networks. Our models predict that a large proportion of the inhibitory network must be perturbed to reliably detect an ISN regime robustly in cortex. We propose that wide-field optogenetic suppression of inhibition under promoters targeting a large fraction of inhibitory neurons may provide a perturbation of sufficient strength to reveal the operating regime of cortex. Our results suggest that detailed computational models of optogenetic perturbations are necessary to interpret the results of experimental paradigms. SIGNIFICANCE STATEMENT Many useful computational mechanisms proposed for cortex require local excitatory recurrence to be very strong, such that local inhibitory feedback is necessary to avoid epileptiform runaway activity (an “inhibition-stabilized network” or “ISN” regime). However, recent experimental results suggest that this regime may not exist in cortex. We simulated activity perturbations in cortical networks of increasing realism and found that, to detect ISN-like properties in cortex, large proportions of the inhibitory population must be perturbed. Current experimental methods for inhibitory perturbation are unlikely to satisfy this requirement, implying that existing experimental observations are inconclusive about the computational regime of cortex. Our results suggest that new experimental designs targeting a majority of inhibitory neurons may be able to resolve this question. PMID:29074575

Surgery in adrenocortical carcinoma: Importance of national cooperation and centralized surgery.

PubMed

Hermsen, Ilse G C; Kerkhofs, Thomas M A; den Butter, Gijsbert; Kievit, Job; van Eijck, Casper H J; Nieveen van Dijkum, Els J M; Haak, Harm R

2012-07-01

The low incidence rate of adrenocortical carcinoma (ACC) requires a multidisciplinary approach in which surgery plays an essential role because complete resection of the primary tumor is the only chance of cure. To improve patient care, insight into operative results within the ACC population is essential. In 2007, a Dutch Adrenal Network Registry was created covering care and outcome of patients treated for ACC in the Netherlands since 1965. Using this database, we performed a study (1) to gain insight into surgical performance in the Netherlands and (2) to compare operative data with international literature. Data on patients treated from 1965 until January 2008 were studied. The following data were collected: age, gender, functionality of the tumor, stage at diagnosis, operative procedure, completeness of surgery, disease recurrence, adjuvant mitotane therapy, and recurrence-free and overall survival (OS). A total of 175 patients were studied, of whom 149 underwent surgery. Patients with complete resection had significantly longer OS times than patients with incomplete resection (P = .010). Patients operated on in a Dutch Adrenal Network center had significantly longer duration of OS in both univariate (P = .011) and multivariate analysis (P = .014). A significantly greater OS was observed for operated stage IV patients compared with nonoperated patients (P = .002). Our data suggest the relevance of national cooperation and centralized surgery in ACC. For selected patients with stage IV disease, surgery can be beneficial in extending survival. On the basis of the retrospective analysis, operative ACC in the Netherlands can and will be improved. Copyright © 2012. Published by Mosby, Inc.

A novel wavelet sequence based on deep bidirectional LSTM network model for ECG signal classification.

PubMed

Yildirim, Özal

2018-05-01

Long-short term memory networks (LSTMs), which have recently emerged in sequential data analysis, are the most widely used type of recurrent neural networks (RNNs) architecture. Progress on the topic of deep learning includes successful adaptations of deep versions of these architectures. In this study, a new model for deep bidirectional LSTM network-based wavelet sequences called DBLSTM-WS was proposed for classifying electrocardiogram (ECG) signals. For this purpose, a new wavelet-based layer is implemented to generate ECG signal sequences. The ECG signals were decomposed into frequency sub-bands at different scales in this layer. These sub-bands are used as sequences for the input of LSTM networks. New network models that include unidirectional (ULSTM) and bidirectional (BLSTM) structures are designed for performance comparisons. Experimental studies have been performed for five different types of heartbeats obtained from the MIT-BIH arrhythmia database. These five types are Normal Sinus Rhythm (NSR), Ventricular Premature Contraction (VPC), Paced Beat (PB), Left Bundle Branch Block (LBBB), and Right Bundle Branch Block (RBBB). The results show that the DBLSTM-WS model gives a high recognition performance of 99.39%. It has been observed that the wavelet-based layer proposed in the study significantly improves the recognition performance of conventional networks. This proposed network structure is an important approach that can be applied to similar signal processing problems. Copyright © 2018 Elsevier Ltd. All rights reserved.

Machine learning in sentiment reconstruction of the simulated stock market

NASA Astrophysics Data System (ADS)

Goykhman, Mikhail; Teimouri, Ali

2018-02-01

In this paper we continue the study of the simulated stock market framework defined by the driving sentiment processes. We focus on the market environment driven by the buy/sell trading sentiment process of the Markov chain type. We apply the methodology of the Hidden Markov Models and the Recurrent Neural Networks to reconstruct the transition probabilities matrix of the Markov sentiment process and recover the underlying sentiment states from the observed stock price behavior. We demonstrate that the Hidden Markov Model can successfully recover the transition probabilities matrix for the hidden sentiment process of the Markov Chain type. We also demonstrate that the Recurrent Neural Network can successfully recover the hidden sentiment states from the observed simulated stock price time series.

Exploring multiple feature combination strategies with a recurrent neural network architecture for off-line handwriting recognition

NASA Astrophysics Data System (ADS)

Mioulet, L.; Bideault, G.; Chatelain, C.; Paquet, T.; Brunessaux, S.

2015-01-01

The BLSTM-CTC is a novel recurrent neural network architecture that has outperformed previous state of the art algorithms in tasks such as speech recognition or handwriting recognition. It has the ability to process long term dependencies in temporal signals in order to label unsegmented data. This paper describes different ways of combining features using a BLSTM-CTC architecture. Not only do we explore the low level combination (feature space combination) but we also explore high level combination (decoding combination) and mid-level (internal system representation combination). The results are compared on the RIMES word database. Our results show that the low level combination works best, thanks to the powerful data modeling of the LSTM neurons.

Automatic temporal segment detection via bilateral long short-term memory recurrent neural networks

NASA Astrophysics Data System (ADS)

Sun, Bo; Cao, Siming; He, Jun; Yu, Lejun; Li, Liandong

2017-03-01

Constrained by the physiology, the temporal factors associated with human behavior, irrespective of facial movement or body gesture, are described by four phases: neutral, onset, apex, and offset. Although they may benefit related recognition tasks, it is not easy to accurately detect such temporal segments. An automatic temporal segment detection framework using bilateral long short-term memory recurrent neural networks (BLSTM-RNN) to learn high-level temporal-spatial features, which synthesizes the local and global temporal-spatial information more efficiently, is presented. The framework is evaluated in detail over the face and body database (FABO). The comparison shows that the proposed framework outperforms state-of-the-art methods for solving the problem of temporal segment detection.

Modeling language and cognition with deep unsupervised learning: a tutorial overview

PubMed Central

Zorzi, Marco; Testolin, Alberto; Stoianov, Ivilin P.

2013-01-01

Deep unsupervised learning in stochastic recurrent neural networks with many layers of hidden units is a recent breakthrough in neural computation research. These networks build a hierarchy of progressively more complex distributed representations of the sensory data by fitting a hierarchical generative model. In this article we discuss the theoretical foundations of this approach and we review key issues related to training, testing and analysis of deep networks for modeling language and cognitive processing. The classic letter and word perception problem of McClelland and Rumelhart (1981) is used as a tutorial example to illustrate how structured and abstract representations may emerge from deep generative learning. We argue that the focus on deep architectures and generative (rather than discriminative) learning represents a crucial step forward for the connectionist modeling enterprise, because it offers a more plausible model of cortical learning as well as a way to bridge the gap between emergentist connectionist models and structured Bayesian models of cognition. PMID:23970869

Modeling language and cognition with deep unsupervised learning: a tutorial overview.

PubMed

Zorzi, Marco; Testolin, Alberto; Stoianov, Ivilin P

2013-01-01

Deep unsupervised learning in stochastic recurrent neural networks with many layers of hidden units is a recent breakthrough in neural computation research. These networks build a hierarchy of progressively more complex distributed representations of the sensory data by fitting a hierarchical generative model. In this article we discuss the theoretical foundations of this approach and we review key issues related to training, testing and analysis of deep networks for modeling language and cognitive processing. The classic letter and word perception problem of McClelland and Rumelhart (1981) is used as a tutorial example to illustrate how structured and abstract representations may emerge from deep generative learning. We argue that the focus on deep architectures and generative (rather than discriminative) learning represents a crucial step forward for the connectionist modeling enterprise, because it offers a more plausible model of cortical learning as well as a way to bridge the gap between emergentist connectionist models and structured Bayesian models of cognition.

A hybrid technique for speech segregation and classification using a sophisticated deep neural network

PubMed Central

Nawaz, Tabassam; Mehmood, Zahid; Rashid, Muhammad; Habib, Hafiz Adnan

2018-01-01

Recent research on speech segregation and music fingerprinting has led to improvements in speech segregation and music identification algorithms. Speech and music segregation generally involves the identification of music followed by speech segregation. However, music segregation becomes a challenging task in the presence of noise. This paper proposes a novel method of speech segregation for unlabelled stationary noisy audio signals using the deep belief network (DBN) model. The proposed method successfully segregates a music signal from noisy audio streams. A recurrent neural network (RNN)-based hidden layer segregation model is applied to remove stationary noise. Dictionary-based fisher algorithms are employed for speech classification. The proposed method is tested on three datasets (TIMIT, MIR-1K, and MusicBrainz), and the results indicate the robustness of proposed method for speech segregation. The qualitative and quantitative analysis carried out on three datasets demonstrate the efficiency of the proposed method compared to the state-of-the-art speech segregation and classification-based methods. PMID:29558485

Monitoring tool usage in surgery videos using boosted convolutional and recurrent neural networks.

PubMed

Al Hajj, Hassan; Lamard, Mathieu; Conze, Pierre-Henri; Cochener, Béatrice; Quellec, Gwenolé

2018-05-09

This paper investigates the automatic monitoring of tool usage during a surgery, with potential applications in report generation, surgical training and real-time decision support. Two surgeries are considered: cataract surgery, the most common surgical procedure, and cholecystectomy, one of the most common digestive surgeries. Tool usage is monitored in videos recorded either through a microscope (cataract surgery) or an endoscope (cholecystectomy). Following state-of-the-art video analysis solutions, each frame of the video is analyzed by convolutional neural networks (CNNs) whose outputs are fed to recurrent neural networks (RNNs) in order to take temporal relationships between events into account. Novelty lies in the way those CNNs and RNNs are trained. Computational complexity prevents the end-to-end training of "CNN+RNN" systems. Therefore, CNNs are usually trained first, independently from the RNNs. This approach is clearly suboptimal for surgical tool analysis: many tools are very similar to one another, but they can generally be differentiated based on past events. CNNs should be trained to extract the most useful visual features in combination with the temporal context. A novel boosting strategy is proposed to achieve this goal: the CNN and RNN parts of the system are simultaneously enriched by progressively adding weak classifiers (either CNNs or RNNs) trained to improve the overall classification accuracy. Experiments were performed in a dataset of 50 cataract surgery videos, where the usage of 21 surgical tools was manually annotated, and a dataset of 80 cholecystectomy videos, where the usage of 7 tools was manually annotated. Very good classification performance are achieved in both datasets: tool usage could be labeled with an average area under the ROC curve of A z =0.9961 and A z =0.9939, respectively, in offline mode (using past, present and future information), and A z =0.9957 and A z =0.9936, respectively, in online mode (using past and present information only). Copyright © 2018 Elsevier B.V. All rights reserved.

Where’s the Noise? Key Features of Spontaneous Activity and Neural Variability Arise through Learning in a Deterministic Network

PubMed Central

Hartmann, Christoph; Lazar, Andreea; Nessler, Bernhard; Triesch, Jochen

2015-01-01

Even in the absence of sensory stimulation the brain is spontaneously active. This background “noise” seems to be the dominant cause of the notoriously high trial-to-trial variability of neural recordings. Recent experimental observations have extended our knowledge of trial-to-trial variability and spontaneous activity in several directions: 1. Trial-to-trial variability systematically decreases following the onset of a sensory stimulus or the start of a motor act. 2. Spontaneous activity states in sensory cortex outline the region of evoked sensory responses. 3. Across development, spontaneous activity aligns itself with typical evoked activity patterns. 4. The spontaneous brain activity prior to the presentation of an ambiguous stimulus predicts how the stimulus will be interpreted. At present it is unclear how these observations relate to each other and how they arise in cortical circuits. Here we demonstrate that all of these phenomena can be accounted for by a deterministic self-organizing recurrent neural network model (SORN), which learns a predictive model of its sensory environment. The SORN comprises recurrently coupled populations of excitatory and inhibitory threshold units and learns via a combination of spike-timing dependent plasticity (STDP) and homeostatic plasticity mechanisms. Similar to balanced network architectures, units in the network show irregular activity and variable responses to inputs. Additionally, however, the SORN exhibits sequence learning abilities matching recent findings from visual cortex and the network’s spontaneous activity reproduces the experimental findings mentioned above. Intriguingly, the network’s behaviour is reminiscent of sampling-based probabilistic inference, suggesting that correlates of sampling-based inference can develop from the interaction of STDP and homeostasis in deterministic networks. We conclude that key observations on spontaneous brain activity and the variability of neural responses can be accounted for by a simple deterministic recurrent neural network which learns a predictive model of its sensory environment via a combination of generic neural plasticity mechanisms. PMID:26714277

Decoding of finger trajectory from ECoG using deep learning.

PubMed

Xie, Ziqian; Schwartz, Odelia; Prasad, Abhishek

2018-06-01

Conventional decoding pipeline for brain-machine interfaces (BMIs) consists of chained different stages of feature extraction, time-frequency analysis and statistical learning models. Each of these stages uses a different algorithm trained in a sequential manner, which makes it difficult to make the whole system adaptive. The goal was to create an adaptive online system with a single objective function and a single learning algorithm so that the whole system can be trained in parallel to increase the decoding performance. Here, we used deep neural networks consisting of convolutional neural networks (CNN) and a special kind of recurrent neural network (RNN) called long short term memory (LSTM) to address these needs. We used electrocorticography (ECoG) data collected by Kubanek et al. The task consisted of individual finger flexions upon a visual cue. Our model combined a hierarchical feature extractor CNN and a RNN that was able to process sequential data and recognize temporal dynamics in the neural data. CNN was used as the feature extractor and LSTM was used as the regression algorithm to capture the temporal dynamics of the signal. We predicted the finger trajectory using ECoG signals and compared results for the least angle regression (LARS), CNN-LSTM, random forest, LSTM model (LSTM_HC, for using hard-coded features) and a decoding pipeline consisting of band-pass filtering, energy extraction, feature selection and linear regression. The results showed that the deep learning models performed better than the commonly used linear model. The deep learning models not only gave smoother and more realistic trajectories but also learned the transition between movement and rest state. This study demonstrated a decoding network for BMI that involved a convolutional and recurrent neural network model. It integrated the feature extraction pipeline into the convolution and pooling layer and used LSTM layer to capture the state transitions. The discussed network eliminated the need to separately train the model at each step in the decoding pipeline. The whole system can be jointly optimized using stochastic gradient descent and is capable of online learning.

Decoding of finger trajectory from ECoG using deep learning

NASA Astrophysics Data System (ADS)

Xie, Ziqian; Schwartz, Odelia; Prasad, Abhishek

2018-06-01

Objective. Conventional decoding pipeline for brain-machine interfaces (BMIs) consists of chained different stages of feature extraction, time-frequency analysis and statistical learning models. Each of these stages uses a different algorithm trained in a sequential manner, which makes it difficult to make the whole system adaptive. The goal was to create an adaptive online system with a single objective function and a single learning algorithm so that the whole system can be trained in parallel to increase the decoding performance. Here, we used deep neural networks consisting of convolutional neural networks (CNN) and a special kind of recurrent neural network (RNN) called long short term memory (LSTM) to address these needs. Approach. We used electrocorticography (ECoG) data collected by Kubanek et al. The task consisted of individual finger flexions upon a visual cue. Our model combined a hierarchical feature extractor CNN and a RNN that was able to process sequential data and recognize temporal dynamics in the neural data. CNN was used as the feature extractor and LSTM was used as the regression algorithm to capture the temporal dynamics of the signal. Main results. We predicted the finger trajectory using ECoG signals and compared results for the least angle regression (LARS), CNN-LSTM, random forest, LSTM model (LSTM_HC, for using hard-coded features) and a decoding pipeline consisting of band-pass filtering, energy extraction, feature selection and linear regression. The results showed that the deep learning models performed better than the commonly used linear model. The deep learning models not only gave smoother and more realistic trajectories but also learned the transition between movement and rest state. Significance. This study demonstrated a decoding network for BMI that involved a convolutional and recurrent neural network model. It integrated the feature extraction pipeline into the convolution and pooling layer and used LSTM layer to capture the state transitions. The discussed network eliminated the need to separately train the model at each step in the decoding pipeline. The whole system can be jointly optimized using stochastic gradient descent and is capable of online learning.

Influence of influenza vaccination on recurrent hospitalization in patients with heart failure.

PubMed

Kaya, H; Beton, O; Acar, G; Temizhan, A; Cavusoğlu, Y; Guray, U; Zoghi, M; Ural, D; Ekmekci, A; Gungor, H; Sari, I; Oguz, D; Yucel, H; Zorlu, A; Yilmaz, M B

2017-05-01

The current study aimed to evaluate the influence of regular annual influenza vaccinations on cardiovascular (CV) death and heart failure-related hospitalizations (HFrH) in stable outpatients with heart failure with reduced ejection fraction. The Turkish research team-HF (TREAT-HF) is a network undertaking multicenter, observational cohort studies in HF. This study is a subgroup analysis of TREAT-HF outpatient cohorts who completed a questionnaire on influenza vaccination status and for whom follow-up data were available. A total of 656 patients with available follow-up data for CV death and HFrH including recurrent hospitalization were included in the study. Patients were classified into two groups: those who received regular influenza vaccination (40 %) and those who did not receive vaccination. During a mean follow-up of 15 ±6 months, 113 (18 %) patients had CV death and 471 (72 %) patients had at least one HFrH. The CV death rate was similar in both groups of patients (16 vs. 19 %, p = 0.37), whereas, HFrH and recurrent HFrH were significantly less frequently encountered in patients who received regular influenza vaccination than in those who did not receive vaccination (43 vs. 92 % and 16 vs. 66 %, p < 0.001, respectively). In a multivariate Cox proportional hazards model - in addition to a few clinical factors - vaccination status (HR = 0.30, 95 % CI = 0.17-0.51, p < 0.001) and graduation from university (HR = 0.35, 95 % CI = 0.17-0.72, p = 0.004) remained independently associated with the risk of recurrent HFrH. Regular influenza vaccination does not influence CV deaths; however, it decreases HFrH including recurrent episodes of HFrH in outpatients with heart failure with reduced ejection fraction.

Investigation of complexity dynamics in a DC glow discharge magnetized plasma using recurrence quantification analysis

DOE Office of Scientific and Technical Information (OSTI.GOV)

Mitra, Vramori; Sarma, Bornali; Sarma, Arun

Recurrence is an ubiquitous feature which provides deep insights into the dynamics of real dynamical systems. A suitable tool for investigating recurrences is recurrence quantification analysis (RQA). It allows, e.g., the detection of regime transitions with respect to varying control parameters. We investigate the complexity of different coexisting nonlinear dynamical regimes of the plasma floating potential fluctuations at different magnetic fields and discharge voltages by using recurrence quantification variables, in particular, DET, L{sub max}, and Entropy. The recurrence analysis reveals that the predictability of the system strongly depends on discharge voltage. Furthermore, the persistent behaviour of the plasma time seriesmore » is characterized by the Detrended fluctuation analysis technique to explore the complexity in terms of long range correlation. The enhancement of the discharge voltage at constant magnetic field increases the nonlinear correlations; hence, the complexity of the system decreases, which corroborates the RQA analysis.« less

Which hemostatic device in thyroid surgery? A network meta-analysis of surgical technologies.

PubMed

Garas, George; Okabayashi, Koji; Ashrafian, Hutan; Shetty, Kunal; Palazzo, Fausto; Tolley, Neil; Darzi, Ara; Athanasiou, Thanos; Zacharakis, Emmanouil

2013-09-01

Energy-based hemostatic devices are increasingly being used in thyroid surgery. However, there are several limitations with regard to the existing evidence and a paucity of guidelines on the subject. The goal of this review is to employ the novel evidence synthesis technique of a network meta-analysis to assess the comparative effectiveness of surgical technologies in thyroid surgery and contribute to enhanced governance in the field of thyroid surgery. Articles published between January 2000 and June 2012 were identified from Embase, Medline, Cochrane Library, and PubMed databases. Randomized controlled trials of any size comparing the use of ultrasonic coagulation (harmonic scalpel) or Ligasure either head-to-head or against the "clamp-and-tie" technique were included. Two reviewers independently critically appraised and extracted the data from each study. The number of patients who experienced postoperative events was extracted in dichotomous format or continuous outcomes. Odds ratios were calculated by a Bayesian network meta-analysis, and metaregression was used for pair-wise comparisons. Indirect and direct comparisons were performed and inconsistency was assessed. Thirty-five randomized controlled trials with 2856 patients were included. Ultrasonic coagulation ranked first (followed by Ligasure and then clamp-and-tie) with the lowest risk of postoperative hypoparathyroidism (odds ratio 1.43 [95% confidence interval (CI) 0.77-2.67] and 0.70 [CI 0.43-1.13], ultrasonic coagulation vs. Ligasure and ultrasonic coagulation vs. clamp-and-tie, respectively), least blood loss (-0.25 [CI -0.84 to -0.35] and -1.22 [CI -1.85 to -0.59]), and drain output (0.28 [CI -0.35 to -0.91] and -0.36 [CI -0.70 to -0.03]). From a health technology viewpoint, ultrasonic coagulation was associated with the shortest operative time (-0.66 [CI -1.17 to -0.14] and -1.29 [CI -1.59 to -1.00]) and hospital stay (-0.28 [CI -0.78 to 0.22] and -0.56 [CI -1.28 to 0.15]). The only exception occurs with the clinically important complication of recurrent laryngeal nerve paralysis, where the reverse trend applies (1.36 [CI 0.25-7.46] and 1.74 [CI 0.94-3.26]). The comparative effectiveness of ultrasonic coagulation in thyroid surgery outcomes seems superior to other techniques with the exception of recurrent laryngeal nerve injury. This network meta-analysis, one of a handful in a surgical field, offers preliminary and robust evidence to guide clinical decisions and policy makers to adopt safer thyroid operations.

Drug treatment rates with beta-blockers and ACE-inhibitors/angiotensin receptor blockers and recurrences in takotsubo cardiomyopathy: A meta-regression analysis.

PubMed

Brunetti, Natale Daniele; Santoro, Francesco; De Gennaro, Luisa; Correale, Michele; Gaglione, Antonio; Di Biase, Matteo

2016-07-01

In a recent paper Singh et al. analyzed the effect of drug treatment on recurrence of takotsubo cardiomyopathy (TTC) in a comprehensive meta-analysis. The study found that recurrence rates were independent of clinic utilization of BB prescription, but inversely correlated with ACEi/ARB prescription: authors therefore conclude that ACEi/ARB rather than BB may reduce risk of recurrence. We aimed to re-analyze data reported in the study, now weighted for populations' size, in a meta-regression analysis. After multiple meta-regression analysis, we found a significant regression between rates of prescription of ACEi and rates of recurrence of TTC; regression was not statistically significant for BBs. On the bases of our re-analysis, we confirm that rates of recurrence of TTC are lower in populations of patients with higher rates of treatment with ACEi/ARB. That could not necessarily imply that ACEi may prevent recurrence of TTC, but barely that, for example, rates of recurrence are lower in cohorts more compliant with therapy or more prescribed with ACEi because more carefully followed. Randomized prospective studies are surely warranted. Copyright © 2016 Elsevier Ireland Ltd. All rights reserved.

Detecting transitions in protein dynamics using a recurrence quantification analysis based bootstrap method.

PubMed

Karain, Wael I

2017-11-28

Proteins undergo conformational transitions over different time scales. These transitions are closely intertwined with the protein's function. Numerous standard techniques such as principal component analysis are used to detect these transitions in molecular dynamics simulations. In this work, we add a new method that has the ability to detect transitions in dynamics based on the recurrences in the dynamical system. It combines bootstrapping and recurrence quantification analysis. We start from the assumption that a protein has a "baseline" recurrence structure over a given period of time. Any statistically significant deviation from this recurrence structure, as inferred from complexity measures provided by recurrence quantification analysis, is considered a transition in the dynamics of the protein. We apply this technique to a 132 ns long molecular dynamics simulation of the β-Lactamase Inhibitory Protein BLIP. We are able to detect conformational transitions in the nanosecond range in the recurrence dynamics of the BLIP protein during the simulation. The results compare favorably to those extracted using the principal component analysis technique. The recurrence quantification analysis based bootstrap technique is able to detect transitions between different dynamics states for a protein over different time scales. It is not limited to linear dynamics regimes, and can be generalized to any time scale. It also has the potential to be used to cluster frames in molecular dynamics trajectories according to the nature of their recurrence dynamics. One shortcoming for this method is the need to have large enough time windows to insure good statistical quality for the recurrence complexity measures needed to detect the transitions.

Convergent evolution of gene networks by single-gene duplications in higher eukaryotes.

PubMed

Amoutzias, Gregory D; Robertson, David L; Oliver, Stephen G; Bornberg-Bauer, Erich

2004-03-01

By combining phylogenetic, proteomic and structural information, we have elucidated the evolutionary driving forces for the gene-regulatory interaction networks of basic helix-loop-helix transcription factors. We infer that recurrent events of single-gene duplication and domain rearrangement repeatedly gave rise to distinct networks with almost identical hub-based topologies, and multiple activators and repressors. We thus provide the first empirical evidence for scale-free protein networks emerging through single-gene duplications, the dominant importance of molecular modularity in the bottom-up construction of complex biological entities, and the convergent evolution of networks.

Recurrent Artificial Neural Networks and Finite State Natural Language Processing.

ERIC Educational Resources Information Center

Moisl, Hermann

It is argued that pessimistic assessments of the adequacy of artificial neural networks (ANNs) for natural language processing (NLP) on the grounds that they have a finite state architecture are unjustified, and that their adequacy in this regard is an empirical issue. First, arguments that counter standard objections to finite state NLP on the…

Birth of an Abstraction: A Dynamical Systems Account of the Discovery of an Elsewhere Principle in a Category Learning Task

ERIC Educational Resources Information Center

Tabor, Whitney; Cho, Pyeong W.; Dankowicz, Harry

2013-01-01

Human participants and recurrent ("connectionist") neural networks were both trained on a categorization system abstractly similar to natural language systems involving irregular ("strong") classes and a default class. Both the humans and the networks exhibited staged learning and a generalization pattern reminiscent of the…

Atypical and Malignant Meningioma: Outcome and Prognostic Factors in 119 Irradiated Patients. A Multicenter, Retrospective Study of the Rare Cancer Network

DOE Office of Scientific and Technical Information (OSTI.GOV)

Pasquier, David; Bijmolt, Stefan; Veninga, Theo

2008-08-01

Purpose: To retrospectively analyze and assess the outcomes and prognostic factors in a large number of patients with atypical and malignant meningiomas. Methods and Materials: Ten academic medical centers participating in this Rare Cancer Network contributed 119 cases of patients with atypical or malignant meningiomas treated with external beam radiotherapy (EBRT) after surgery or for recurrence. Eligibility criteria were histologically proven atypical or anaplastic (malignant) meningioma (World Health Organization Grade 2 and 3) treated with fractionated EBRT after initial resection or for recurrence, and age >18 years. Sex ratio (male/female) was 1.3, and mean ({+-}SD) age was 57.6 {+-} 12more » years. Surgery was macroscopically complete (Simpson Grades 1-3) in 71% of patients; histology was atypical and malignant in 69% and 31%, respectively. Mean dose of EBRT was 54.6 {+-} 5.1 Gy (range, 40-66 Gy). Median follow-up was 4.1 years. Results: The 5- and 10-year actuarial overall survival rates were 65% and 51%, respectively, and were significantly influenced by age >60 years (p = 0.005), Karnofsky performance status (KPS) (p = 0.01), and high mitotic rate (p = 0.047) on univariate analysis. On multivariate analysis age >60 years (p = 0.001) and high mitotic rate (p = 0.02) remained significant adverse prognostic factors. The 5- and 10-year disease-free survival rates were 58% and 48%, respectively, and were significantly influenced by KPS (p 0.04) and high mitotic rate (p = 0.003) on univariate analysis. On multivariate analysis only high mitotic rate (p = 0.003) remained a significant prognostic factor. Conclusions: In this multicenter retrospective study, age, KPS, and mitotic rate influenced outcome. Multicenter prospective studies are necessary to clarify the management and prognostic factors of such a rare disease.« less

The frequency preference of neurons and synapses in a recurrent oscillatory network.

PubMed

Tseng, Hua-an; Martinez, Diana; Nadim, Farzan

2014-09-17

A variety of neurons and synapses shows a maximal response at a preferred frequency, generally considered to be important in shaping network activity. We are interested in whether all neurons and synapses in a recurrent oscillatory network can have preferred frequencies and, if so, whether these frequencies are the same or correlated, and whether they influence the network activity. We address this question using identified neurons in the pyloric network of the crab Cancer borealis. Previous work has shown that the pyloric pacemaker neurons exhibit membrane potential resonance whose resonance frequency is correlated with the network frequency. The follower lateral pyloric (LP) neuron makes reciprocally inhibitory synapses with the pacemakers. We find that LP shows resonance at a higher frequency than the pacemakers and the network frequency falls between the two. We also find that the reciprocal synapses between the pacemakers and LP have preferred frequencies but at significantly lower values. The preferred frequency of the LP to pacemaker synapse is correlated with the presynaptic preferred frequency, which is most pronounced when the peak voltage of the LP waveform is within the dynamic range of the synaptic activation curve and a shift in the activation curve by the modulatory neuropeptide proctolin shifts the frequency preference. Proctolin also changes the power of the LP neuron resonance without significantly changing the resonance frequency. These results indicate that different neuron types and synapses in a network may have distinct preferred frequencies, which are subject to neuromodulation and may interact to shape network oscillations. Copyright © 2014 the authors 0270-6474/14/3412933-13$15.00/0.

A Fixed Point Theorem in Weak Topology for Successively Recurrent System of Set-Valued Mapping Equations and Its Applications

NASA Astrophysics Data System (ADS)

Horiuchi, Kazuo

Let us introduce n (≥ 2) mappings fi(i = 1, …, n ≡ 0) defined on reflexive real Banach spaces Xi-1 and let fi : Xi-1 → Yi be completely continuous on bounded convex closed subsets X_{i-1}^{(0)} \\\\subset X_{i-1}. Moreover, let us introduce n set-valued mappings F_i : X_{i-1} \\\\times Y_i \\\\to {\\\\cal F}_c(X_i) (the family of all non-empty compact subsets of Xi), (i=1, …, n ≡ 0). Here, we have a fixed point theorem in weak topology on the successively recurrent system of set-valued mapping equations: xi ∈ Fi(xi-1, fi(xi-1)), (i=1, …, n ≡ 0). This theorem can be applied immediately to analysis of the availability of system of circular networks of channels undergone by uncertain fluctuations and to evaluation of the tolerability of behaviors of those systems.

Cross-Participant EEG-Based Assessment of Cognitive Workload Using Multi-Path Convolutional Recurrent Neural Networks.

PubMed

Hefron, Ryan; Borghetti, Brett; Schubert Kabban, Christine; Christensen, James; Estepp, Justin

2018-04-26

Applying deep learning methods to electroencephalograph (EEG) data for cognitive state assessment has yielded improvements over previous modeling methods. However, research focused on cross-participant cognitive workload modeling using these techniques is underrepresented. We study the problem of cross-participant state estimation in a non-stimulus-locked task environment, where a trained model is used to make workload estimates on a new participant who is not represented in the training set. Using experimental data from the Multi-Attribute Task Battery (MATB) environment, a variety of deep neural network models are evaluated in the trade-space of computational efficiency, model accuracy, variance and temporal specificity yielding three important contributions: (1) The performance of ensembles of individually-trained models is statistically indistinguishable from group-trained methods at most sequence lengths. These ensembles can be trained for a fraction of the computational cost compared to group-trained methods and enable simpler model updates. (2) While increasing temporal sequence length improves mean accuracy, it is not sufficient to overcome distributional dissimilarities between individuals’ EEG data, as it results in statistically significant increases in cross-participant variance. (3) Compared to all other networks evaluated, a novel convolutional-recurrent model using multi-path subnetworks and bi-directional, residual recurrent layers resulted in statistically significant increases in predictive accuracy and decreases in cross-participant variance.

An automatic microseismic or acoustic emission arrival identification scheme with deep recurrent neural networks

NASA Astrophysics Data System (ADS)

Zheng, Jing; Lu, Jiren; Peng, Suping; Jiang, Tianqi

2018-02-01

The conventional arrival pick-up algorithms cannot avoid the manual modification of the parameters for the simultaneous identification of multiple events under different signal-to-noise ratios (SNRs). Therefore, in order to automatically obtain the arrivals of multiple events with high precision under different SNRs, in this study an algorithm was proposed which had the ability to pick up the arrival of microseismic or acoustic emission events based on deep recurrent neural networks. The arrival identification was performed using two important steps, which included a training phase and a testing phase. The training process was mathematically modelled by deep recurrent neural networks using Long Short-Term Memory architecture. During the testing phase, the learned weights were utilized to identify the arrivals through the microseismic/acoustic emission data sets. The data sets were obtained by rock physics experiments of the acoustic emission. In order to obtain the data sets under different SNRs, this study added random noise to the raw experiments' data sets. The results showed that the outcome of the proposed method was able to attain an above 80 per cent hit-rate at SNR 0 dB, and an approximately 70 per cent hit-rate at SNR -5 dB, with an absolute error in 10 sampling points. These results indicated that the proposed method had high selection precision and robustness.

An adaptive PID like controller using mix locally recurrent neural network for robotic manipulator with variable payload.

PubMed

Sharma, Richa; Kumar, Vikas; Gaur, Prerna; Mittal, A P

2016-05-01

Being complex, non-linear and coupled system, the robotic manipulator cannot be effectively controlled using classical proportional-integral-derivative (PID) controller. To enhance the effectiveness of the conventional PID controller for the nonlinear and uncertain systems, gains of the PID controller should be conservatively tuned and should adapt to the process parameter variations. In this work, a mix locally recurrent neural network (MLRNN) architecture is investigated to mimic a conventional PID controller which consists of at most three hidden nodes which act as proportional, integral and derivative node. The gains of the mix locally recurrent neural network based PID (MLRNNPID) controller scheme are initialized with a newly developed cuckoo search algorithm (CSA) based optimization method rather than assuming randomly. A sequential learning based least square algorithm is then investigated for the on-line adaptation of the gains of MLRNNPID controller. The performance of the proposed controller scheme is tested against the plant parameters uncertainties and external disturbances for both links of the two link robotic manipulator with variable payload (TL-RMWVP). The stability of the proposed controller is analyzed using Lyapunov stability criteria. A performance comparison is carried out among MLRNNPID controller, CSA optimized NNPID (OPTNNPID) controller and CSA optimized conventional PID (OPTPID) controller in order to establish the effectiveness of the MLRNNPID controller. Copyright © 2016 ISA. Published by Elsevier Ltd. All rights reserved.

Cross-Participant EEG-Based Assessment of Cognitive Workload Using Multi-Path Convolutional Recurrent Neural Networks

PubMed Central

Hefron, Ryan; Borghetti, Brett; Schubert Kabban, Christine; Christensen, James; Estepp, Justin

2018-01-01

Applying deep learning methods to electroencephalograph (EEG) data for cognitive state assessment has yielded improvements over previous modeling methods. However, research focused on cross-participant cognitive workload modeling using these techniques is underrepresented. We study the problem of cross-participant state estimation in a non-stimulus-locked task environment, where a trained model is used to make workload estimates on a new participant who is not represented in the training set. Using experimental data from the Multi-Attribute Task Battery (MATB) environment, a variety of deep neural network models are evaluated in the trade-space of computational efficiency, model accuracy, variance and temporal specificity yielding three important contributions: (1) The performance of ensembles of individually-trained models is statistically indistinguishable from group-trained methods at most sequence lengths. These ensembles can be trained for a fraction of the computational cost compared to group-trained methods and enable simpler model updates. (2) While increasing temporal sequence length improves mean accuracy, it is not sufficient to overcome distributional dissimilarities between individuals’ EEG data, as it results in statistically significant increases in cross-participant variance. (3) Compared to all other networks evaluated, a novel convolutional-recurrent model using multi-path subnetworks and bi-directional, residual recurrent layers resulted in statistically significant increases in predictive accuracy and decreases in cross-participant variance. PMID:29701668

Transcriptome from circulating cells suggests dysregulated pathways associated with long-term recurrent events following first-time myocardial infarction.

PubMed

Suresh, Rahul; Li, Xing; Chiriac, Anca; Goel, Kashish; Terzic, Andre; Perez-Terzic, Carmen; Nelson, Timothy J

2014-09-01

Whole-genome gene expression analysis has been successfully utilized to diagnose, prognosticate, and identify potential therapeutic targets for high-risk cardiovascular diseases. However, the feasibility of this approach to identify outcome-related genes and dysregulated pathways following first-time myocardial infarction (AMI) remains unknown and may offer a novel strategy to detect affected expressome networks that predict long-term outcome. Whole-genome expression microarray on blood samples from normal cardiac function controls (n=21) and first-time AMI patients (n=31) within 48-hours post-MI revealed expected differential gene expression profiles enriched for inflammation and immune-response pathways. To determine molecular signatures at the time of AMI associated with long-term outcomes, transcriptional profiles from sub-groups of AMI patients with (n=5) or without (n=22) any recurrent events over an 18-month follow-up were compared. This analysis identified 559 differentially-expressed genes. Bioinformatic analysis of this differential gene-set for associated pathways revealed 1) increasing disease severity in AMI patients is associated with a decreased expression of genes involved in the developmental epithelial-to-mesenchymal transition pathway, and 2) modulation of cholesterol transport genes that include ABCA1, CETP, APOA1, and LDLR is associated with clinical outcome. Differentially regulated genes and modulated pathways were identified that were associated with recurrent cardiovascular outcomes in first-time AMI patients. This cell-based approach for risk stratification in AMI could represent a novel, non-invasive platform to anticipate modifiable pathways and therapeutic targets to optimize long-term outcome for AMI patients and warrants further study to determine the role of metabolic remodeling and regenerative processes required for optimal outcomes. Copyright © 2014 Elsevier Ltd. All rights reserved.

Nonlinear Recurrent Neural Network Predictive Control for Energy Distribution of a Fuel Cell Powered Robot

PubMed Central

Chen, Qihong; Long, Rong; Quan, Shuhai

2014-01-01

This paper presents a neural network predictive control strategy to optimize power distribution for a fuel cell/ultracapacitor hybrid power system of a robot. We model the nonlinear power system by employing time variant auto-regressive moving average with exogenous (ARMAX), and using recurrent neural network to represent the complicated coefficients of the ARMAX model. Because the dynamic of the system is viewed as operating- state- dependent time varying local linear behavior in this frame, a linear constrained model predictive control algorithm is developed to optimize the power splitting between the fuel cell and ultracapacitor. The proposed algorithm significantly simplifies implementation of the controller and can handle multiple constraints, such as limiting substantial fluctuation of fuel cell current. Experiment and simulation results demonstrate that the control strategy can optimally split power between the fuel cell and ultracapacitor, limit the change rate of the fuel cell current, and so as to extend the lifetime of the fuel cell. PMID:24707206

Critical regimes driven by recurrent mobility patterns of reaction-diffusion processes in networks

NASA Astrophysics Data System (ADS)

Gómez-Gardeñes, J.; Soriano-Paños, D.; Arenas, A.

2018-04-01

Reaction-diffusion processes1 have been widely used to study dynamical processes in epidemics2-4 and ecology5 in networked metapopulations. In the context of epidemics6, reaction processes are understood as contagions within each subpopulation (patch), while diffusion represents the mobility of individuals between patches. Recently, the characteristics of human mobility7, such as its recurrent nature, have been proven crucial to understand the phase transition to endemic epidemic states8,9. Here, by developing a framework able to cope with the elementary epidemic processes, the spatial distribution of populations and the commuting mobility patterns, we discover three different critical regimes of the epidemic incidence as a function of these parameters. Interestingly, we reveal a regime of the reaction-diffussion process in which, counter-intuitively, mobility is detrimental to the spread of disease. We analytically determine the precise conditions for the emergence of any of the three possible critical regimes in real and synthetic networks.

Algorithm for Training a Recurrent Multilayer Perceptron

NASA Technical Reports Server (NTRS)

Parlos, Alexander G.; Rais, Omar T.; Menon, Sunil K.; Atiya, Amir F.

2004-01-01

An improved algorithm has been devised for training a recurrent multilayer perceptron (RMLP) for optimal performance in predicting the behavior of a complex, dynamic, and noisy system multiple time steps into the future. [An RMLP is a computational neural network with self-feedback and cross-talk (both delayed by one time step) among neurons in hidden layers]. Like other neural-network-training algorithms, this algorithm adjusts network biases and synaptic-connection weights according to a gradient-descent rule. The distinguishing feature of this algorithm is a combination of global feedback (the use of predictions as well as the current output value in computing the gradient at each time step) and recursiveness. The recursive aspect of the algorithm lies in the inclusion of the gradient of predictions at each time step with respect to the predictions at the preceding time step; this recursion enables the RMLP to learn the dynamics. It has been conjectured that carrying the recursion to even earlier time steps would enable the RMLP to represent a noisier, more complex system.

Sensorless control for permanent magnet synchronous motor using a neural network based adaptive estimator

NASA Astrophysics Data System (ADS)

Kwon, Chung-Jin; Kim, Sung-Joong; Han, Woo-Young; Min, Won-Kyoung

2005-12-01

The rotor position and speed estimation of permanent-magnet synchronous motor(PMSM) was dealt with. By measuring the phase voltages and currents of the PMSM drive, two diagonally recurrent neural network(DRNN) based observers, a neural current observer and a neural velocity observer were developed. DRNN which has self-feedback of the hidden neurons ensures that the outputs of DRNN contain the whole past information of the system even if the inputs of DRNN are only the present states and inputs of the system. Thus the structure of DRNN may be simpler than that of feedforward and fully recurrent neural networks. If the backpropagation method was used for the training of the DRNN the problem of slow convergence arise. In order to reduce this problem, recursive prediction error(RPE) based learning method for the DRNN was presented. The simulation results show that the proposed approach gives a good estimation of rotor speed and position, and RPE based training has requires a shorter computation time compared to backpropagation based training.

Combinatorial Optimization by Amoeba-Based Neurocomputer with Chaotic Dynamics

NASA Astrophysics Data System (ADS)

Aono, Masashi; Hirata, Yoshito; Hara, Masahiko; Aihara, Kazuyuki

We demonstrate a computing system based on an amoeba of a true slime mold Physarum capable of producing rich spatiotemporal oscillatory behavior. Our system operates as a neurocomputer because an optical feedback control in accordance with a recurrent neural network algorithm leads the amoeba's photosensitive branches to search for a stable configuration concurrently. We show our system's capability of solving the traveling salesman problem. Furthermore, we apply various types of nonlinear time series analysis to the amoeba's oscillatory behavior in the problem-solving process. The results suggest that an individual amoeba might be characterized as a set of coupled chaotic oscillators.

Second-Order Analysis of Semiparametric Recurrent Event Processes

PubMed Central

Guan, Yongtao

2011-01-01

Summary A typical recurrent event dataset consists of an often large number of recurrent event processes, each of which contains multiple event times observed from an individual during a followup period. Such data have become increasingly available in medical and epidemiological studies. In this paper, we introduce novel procedures to conduct second-order analysis for a flexible class of semiparametric recurrent event processes. Such an analysis can provide useful information regarding the dependence structure within each recurrent event process. Specifically, we will use the proposed procedures to test whether the individual recurrent event processes are all Poisson processes and to suggest sensible alternative models for them if they are not. We apply these procedures to a well-known recurrent event dataset on chronic granulomatous disease and an epidemiological dataset on Meningococcal disease cases in Merseyside, UK to illustrate their practical value. PMID:21361885

Familiarity Detection is an Intrinsic Property of Cortical Microcircuits with Bidirectional Synaptic Plasticity.

PubMed

Zhang, Xiaoyu; Ju, Han; Penney, Trevor B; VanDongen, Antonius M J

2017-01-01

Humans instantly recognize a previously seen face as "familiar." To deepen our understanding of familiarity-novelty detection, we simulated biologically plausible neural network models of generic cortical microcircuits consisting of spiking neurons with random recurrent synaptic connections. NMDA receptor (NMDAR)-dependent synaptic plasticity was implemented to allow for unsupervised learning and bidirectional modifications. Network spiking activity evoked by sensory inputs consisting of face images altered synaptic efficacy, which resulted in the network responding more strongly to a previously seen face than a novel face. Network size determined how many faces could be accurately recognized as familiar. When the simulated model became sufficiently complex in structure, multiple familiarity traces could be retained in the same network by forming partially-overlapping subnetworks that differ slightly from each other, thereby resulting in a high storage capacity. Fisher's discriminant analysis was applied to identify critical neurons whose spiking activity predicted familiar input patterns. Intriguingly, as sensory exposure was prolonged, the selected critical neurons tended to appear at deeper layers of the network model, suggesting recruitment of additional circuits in the network for incremental information storage. We conclude that generic cortical microcircuits with bidirectional synaptic plasticity have an intrinsic ability to detect familiar inputs. This ability does not require a specialized wiring diagram or supervision and can therefore be expected to emerge naturally in developing cortical circuits.

Familiarity Detection is an Intrinsic Property of Cortical Microcircuits with Bidirectional Synaptic Plasticity

PubMed Central

2017-01-01

Abstract Humans instantly recognize a previously seen face as “familiar.” To deepen our understanding of familiarity-novelty detection, we simulated biologically plausible neural network models of generic cortical microcircuits consisting of spiking neurons with random recurrent synaptic connections. NMDA receptor (NMDAR)-dependent synaptic plasticity was implemented to allow for unsupervised learning and bidirectional modifications. Network spiking activity evoked by sensory inputs consisting of face images altered synaptic efficacy, which resulted in the network responding more strongly to a previously seen face than a novel face. Network size determined how many faces could be accurately recognized as familiar. When the simulated model became sufficiently complex in structure, multiple familiarity traces could be retained in the same network by forming partially-overlapping subnetworks that differ slightly from each other, thereby resulting in a high storage capacity. Fisher’s discriminant analysis was applied to identify critical neurons whose spiking activity predicted familiar input patterns. Intriguingly, as sensory exposure was prolonged, the selected critical neurons tended to appear at deeper layers of the network model, suggesting recruitment of additional circuits in the network for incremental information storage. We conclude that generic cortical microcircuits with bidirectional synaptic plasticity have an intrinsic ability to detect familiar inputs. This ability does not require a specialized wiring diagram or supervision and can therefore be expected to emerge naturally in developing cortical circuits. PMID:28534043

New molecular abnormalities and clonal architecture in AML: from reciprocal translocations to whole-genome sequencing.

PubMed

Graubert, Timothy A; Brunner, Andrew M; Fathi, Amir T

2014-01-01

Acute myeloid leukemia (AML) is characterized by recurrent genetic alterations, including amplifications, deletions, rearrangements, and point mutations. Clinically, these lesions can be used to stratify patients into categories of risk, which directs further clinical management and prognostication. Patient risk categories were first described based on recurrent karyotypic abnormalities; most patients with AML, however, fall into intermediate cytogenetic risk, the majority harboring a normal karyotype. Subsequently, identification of recurrently mutated genes, including FLT3, NPM1, and CEBPA, allowed further stratification of patients with a normal karyotype. More extensive genomic and epigenomic analysis of AML samples has expanded the number of known molecular alterations present in this disease. The further understanding of this mutational landscape has shed light into the pathogenesis of AML. AML arises in a founding clone that often gives rise to subclones. Clonal evolution is a feature of the natural history of the disease but may also be influenced by the selective pressure of chemotherapy. The complex network of genetic and epigenetic alterations in this disease has yielded numerous new targets for intervention. In the future, further understanding of this mutational framework, along with the development of novel therapeutic targets, may lead to improved outcomes for patients with AML.

Economic analysis of routine neuromonitoring of recurrent laryngeal nerve in total thyroidectomy.

PubMed

Sanabria, Álvaro; Ramírez, Adonis

2015-01-01

Thyroidectomy is a common surgery. Routine searching of the recurrent laryngeal nerve is the most important strategy to avoid palsy. Neuromonitoring has been recommended to decrease recurrent laryngeal nerve palsy. To assess if neuromonitoring of recurrent laryngeal nerve during thyroidectomy is cost-effective in a developing country. We designed a decision analysis to assess the cost-effectiveness of recurrent laryngeal nerve neuromonitoring. For probabilities, we used data from a meta-analysis. Utility was measured using preference values. We considered direct costs. We conducted a deterministic and a probabilistic analysis. We did not find differences in utility between arms. The frequency of recurrent laryngeal nerve injury was 1% in the neuromonitor group and 1.6% for the standard group. Thyroidectomy without monitoring was the less expensive alternative. The incremental cost-effectiveness ratio was COP$ 9,112,065. Routine neuromonitoring in total thyroidectomy with low risk of recurrent laryngeal nerve injury is neither cost-useful nor cost-effective in the Colombian health system.

Different propagation speeds of recalled sequences in plastic spiking neural networks

NASA Astrophysics Data System (ADS)

Huang, Xuhui; Zheng, Zhigang; Hu, Gang; Wu, Si; Rasch, Malte J.

2015-03-01

Neural networks can generate spatiotemporal patterns of spike activity. Sequential activity learning and retrieval have been observed in many brain areas, and e.g. is crucial for coding of episodic memory in the hippocampus or generating temporal patterns during song production in birds. In a recent study, a sequential activity pattern was directly entrained onto the neural activity of the primary visual cortex (V1) of rats and subsequently successfully recalled by a local and transient trigger. It was observed that the speed of activity propagation in coordinates of the retinotopically organized neural tissue was constant during retrieval regardless how the speed of light stimulation sweeping across the visual field during training was varied. It is well known that spike-timing dependent plasticity (STDP) is a potential mechanism for embedding temporal sequences into neural network activity. How training and retrieval speeds relate to each other and how network and learning parameters influence retrieval speeds, however, is not well described. We here theoretically analyze sequential activity learning and retrieval in a recurrent neural network with realistic synaptic short-term dynamics and STDP. Testing multiple STDP rules, we confirm that sequence learning can be achieved by STDP. However, we found that a multiplicative nearest-neighbor (NN) weight update rule generated weight distributions and recall activities that best matched the experiments in V1. Using network simulations and mean-field analysis, we further investigated the learning mechanisms and the influence of network parameters on recall speeds. Our analysis suggests that a multiplicative STDP rule with dominant NN spike interaction might be implemented in V1 since recall speed was almost constant in an NMDA-dominant regime. Interestingly, in an AMPA-dominant regime, neural circuits might exhibit recall speeds that instead follow the change in stimulus speeds. This prediction could be tested in experiments.

Alterations of network synchrony after epileptic seizures: An analysis of post-ictal intracranial recordings in pediatric epilepsy patients.

PubMed

Tomlinson, Samuel B; Khambhati, Ankit N; Bermudez, Camilo; Kamens, Rebecca M; Heuer, Gregory G; Porter, Brenda E; Marsh, Eric D

2018-07-01

Post-ictal EEG alterations have been identified in studies of intracranial recordings, but the clinical significance of post-ictal EEG activity is undetermined. The purpose of this study was to examine the relationship between peri-ictal EEG activity, surgical outcome, and extent of seizure propagation in a sample of pediatric epilepsy patients. Intracranial EEG recordings were obtained from 19 patients (mean age = 11.4 years, range = 3-20 years) with 57 seizures used for analysis (mean = 3.0 seizures per patient). For each seizure, 3-min segments were extracted from adjacent pre-ictal and post-ictal epochs. To compare physiology of the epileptic network between epochs, we calculated the relative delta power (Δ) using discrete Fourier transformation and constructed functional networks based on broadband connectivity (conn). We investigated differences between the pre-ictal (Δ pre , conn pre ) and post-ictal (Δ post , conn post ) segments in focal-network (i.e., confined to seizure onset zone) versus distributed-network (i.e., diffuse ictal propagation) seizures. Distributed-network (DN) seizures exhibited increased post-ictal delta power and global EEG connectivity compared to focal-network (FN) seizures. Following DN seizures, patients with seizure-free outcomes exhibited a 14.7% mean increase in delta power and an 8.3% mean increase in global connectivity compared to pre-ictal baseline, which was dramatically less than values observed among seizure-persistent patients (29.6% and 47.1%, respectively). Post-ictal differences between DN and FN seizures correlate with post-operative seizure persistence. We hypothesize that post-ictal deactivation of subcortical nuclei recruited during seizure propagation may account for this result while lending insights into mechanisms of post-operative seizure recurrence. Copyright © 2018 Elsevier B.V. All rights reserved.

How Deep Neural Networks Can Improve Emotion Recognition on Video Data

DTIC Science & Technology

2016-09-25

HOW DEEP NEURAL NETWORKS CAN IMPROVE EMOTION RECOGNITION ON VIDEO DATA Pooya Khorrami1 , Tom Le Paine1, Kevin Brady2, Charlie Dagli2, Thomas S...this work, we present a system that per- forms emotion recognition on video data using both con- volutional neural networks (CNNs) and recurrent...neural net- works (RNNs). We present our findings on videos from the Audio/Visual+Emotion Challenge (AV+EC2015). In our experiments, we analyze the effects

Real time unsupervised learning of visual stimuli in neuromorphic VLSI systems

PubMed Central

Giulioni, Massimiliano; Corradi, Federico; Dante, Vittorio; del Giudice, Paolo

2015-01-01

Neuromorphic chips embody computational principles operating in the nervous system, into microelectronic devices. In this domain it is important to identify computational primitives that theory and experiments suggest as generic and reusable cognitive elements. One such element is provided by attractor dynamics in recurrent networks. Point attractors are equilibrium states of the dynamics (up to fluctuations), determined by the synaptic structure of the network; a ‘basin’ of attraction comprises all initial states leading to a given attractor upon relaxation, hence making attractor dynamics suitable to implement robust associative memory. The initial network state is dictated by the stimulus, and relaxation to the attractor state implements the retrieval of the corresponding memorized prototypical pattern. In a previous work we demonstrated that a neuromorphic recurrent network of spiking neurons and suitably chosen, fixed synapses supports attractor dynamics. Here we focus on learning: activating on-chip synaptic plasticity and using a theory-driven strategy for choosing network parameters, we show that autonomous learning, following repeated presentation of simple visual stimuli, shapes a synaptic connectivity supporting stimulus-selective attractors. Associative memory develops on chip as the result of the coupled stimulus-driven neural activity and ensuing synaptic dynamics, with no artificial separation between learning and retrieval phases. PMID:26463272

Jordan recurrent neural network versus IHACRES in modelling daily streamflows

NASA Astrophysics Data System (ADS)

Carcano, Elena Carla; Bartolini, Paolo; Muselli, Marco; Piroddi, Luigi

2008-12-01

SummaryA study of possible scenarios for modelling streamflow data from daily time series, using artificial neural networks (ANNs), is presented. Particular emphasis is devoted to the reconstruction of drought periods where water resource management and control are most critical. This paper considers two connectionist models: a feedforward multilayer perceptron (MLP) and a Jordan recurrent neural network (JNN), comparing network performance on real world data from two small catchments (192 and 69 km 2 in size) with irregular and torrential regimes. Several network configurations are tested to ensure a good combination of input features (rainfall and previous streamflow data) that capture the variability of the physical processes at work. Tapped delayed line (TDL) and memory effect techniques are introduced to recognize and reproduce temporal dependence. Results show a poor agreement when using TDL only, but a remarkable improvement can be obtained with JNN and its memory effect procedures, which are able to reproduce the system memory over a catchment in a more effective way. Furthermore, the IHACRES conceptual model, which relies on both rainfall and temperature input data, is introduced for comparative study. The results suggest that when good input data is unavailable, metric models perform better than conceptual ones and, in general, it is difficult to justify substantial conceptualization of complex processes.

Cost-effectiveness of apixaban versus low molecular weight heparin/vitamin k antagonist for the treatment of venous thromboembolism and the prevention of recurrences.

PubMed

Lanitis, Tereza; Leipold, Robert; Hamilton, Melissa; Rublee, Dale; Quon, Peter; Browne, Chantelle; Cohen, Alexander T

2017-01-23

Prior analyses beyond clinical trials are yet to evaluate the projected lifetime benefit of apixaban treatment compared to low-molecular-weight heparin (LMWH)/vitamin K antagonist (VKA) for treatment of venous thromboembolism (VTE) and prevention of recurrences. The objective of this study is to assess the cost-effectiveness of initial plus extended treatment with apixaban versus LMWH/VKA for either initial treatment only or initial plus extended treatment. A Markov cohort model was developed to evaluate the lifetime clinical and economic impact of treatment of VTE and prevention of recurrences with apixaban (starting at 10 mg BID for 1 week, then 5 mg BID for 6 months, then 2.5 mg BID for an additional 12 months) versus LMWH/VKA for 6 months and either no further treatment or extended treatment with VKA for an additional 12 months. Clinical event rates to inform the model were taken from the AMPLIFY and AMPLIFY-EXT trials and a network meta-analysis. Background mortality rates, costs, and utilities were obtained from published sources. The analysis was conducted from the perspective of the United Kingdom National Health Service. The evaluated outcomes included the number of events avoided in a 1000-patient cohort, total costs, life-years, quality-adjusted life-years (QALYs), and cost per QALY gained. Initial plus extended treatment with apixaban was superior to both treatment durations of LMWH/VKA in reducing the number of bleeding events, and was superior to initial LMWH/VKA for 6 months followed by no therapy, in reducing VTE recurrences. Apixaban treatment was cost-effective compared to 6-month treatment with LMWH/VKA at an incremental cost-effectiveness ratio (ICER) of £6692 per QALY. When initial LMWH/VKA was followed by further VKA therapy for an additional 12 months (i.e., total treatment duration of 18 months), apixaban was cost-effective at an ICER of £8528 per QALY gained. Sensitivity analysis suggested these findings were robust over a wide range of inputs and scenarios for the model. In the UK, initial plus extended treatment with apixaban for treatment of VTE and prevention of recurrences appears to be economical and a clinically effective alternative to LMWH/VKA, whether used for initial or initial plus extended treatment.

Recurrent-neural-network-based Boolean factor analysis and its application to word clustering.

PubMed

Frolov, Alexander A; Husek, Dusan; Polyakov, Pavel Yu

2009-07-01

The objective of this paper is to introduce a neural-network-based algorithm for word clustering as an extension of the neural-network-based Boolean factor analysis algorithm (Frolov , 2007). It is shown that this extended algorithm supports even the more complex model of signals that are supposed to be related to textual documents. It is hypothesized that every topic in textual data is characterized by a set of words which coherently appear in documents dedicated to a given topic. The appearance of each word in a document is coded by the activity of a particular neuron. In accordance with the Hebbian learning rule implemented in the network, sets of coherently appearing words (treated as factors) create tightly connected groups of neurons, hence, revealing them as attractors of the network dynamics. The found factors are eliminated from the network memory by the Hebbian unlearning rule facilitating the search of other factors. Topics related to the found sets of words can be identified based on the words' semantics. To make the method complete, a special technique based on a Bayesian procedure has been developed for the following purposes: first, to provide a complete description of factors in terms of component probability, and second, to enhance the accuracy of classification of signals to determine whether it contains the factor. Since it is assumed that every word may possibly contribute to several topics, the proposed method might be related to the method of fuzzy clustering. In this paper, we show that the results of Boolean factor analysis and fuzzy clustering are not contradictory, but complementary. To demonstrate the capabilities of this attempt, the method is applied to two types of textual data on neural networks in two different languages. The obtained topics and corresponding words are at a good level of agreement despite the fact that identical topics in Russian and English conferences contain different sets of keywords.

A new switching control for finite-time synchronization of memristor-based recurrent neural networks.

PubMed

Gao, Jie; Zhu, Peiyong; Alsaedi, Ahmed; Alsaadi, Fuad E; Hayat, Tasawar

2017-02-01

In this paper, finite-time synchronization (FTS) of memristor-based recurrent neural networks (MNNs) with time-varying delays is investigated by designing a new switching controller. First, by using the differential inclusions theory and set-valued maps, sufficient conditions to ensure FTS of MNNs are obtained under the two cases of 0<α<1 and α=0, and it is derived that α=0 is the critical value of 0<α<1. Next, it is discussed deeply on the relation between the parameter α and the synchronization time. Then, a new controller with a switching parameter α is designed which can shorten the synchronization time. Finally, some numerical simulation examples are provided to illustrate the effectiveness of the proposed results. Copyright © 2016 Elsevier Ltd. All rights reserved.

Multi-institutional Outcomes of Endoscopic Management of Stricture Recurrence after Bulbar Urethroplasty.

PubMed

Sukumar, Shyam; Elliott, Sean P; Myers, Jeremy B; Voelzke, Bryan B; Smith, Thomas G; Carolan, Alexandra Mc; Maidaa, Michael; Vanni, Alex J; Breyer, Benjamin N; Erickson, Bradley A

2018-05-03

Approximately 10-20% of patients will have a recurrence after urethroplasty. Initial management of these recurrences is often with urethral dilation (UD) or direct vision internal urethrotomy (DVIU). In the current study, we describe outcomes of endoscopic management of stricture recurrence after bulbar urethroplasty. We retrospectively reviewed bulbar urethroplasty data from 5 surgeons from the Trauma and Urologic Reconstruction Network of Surgeons. Men who underwent UD or DVIU for urethroplasty recurrence were identified. Recurrence was defined as inability to pass a 17Fr cystoscope through the area of reconstruction. The primary outcome was the success rate of recurrence management. Comparisons were made between UD and DVIU and then between endoscopic management of recurrences after excision and primary anastomosis urethroplasty (EPA) versus substitutional repairs using time-to-event statistics. There were 53 men with recurrence that were initially managed endoscopically. Median time to urethral stricture recurrence after urethroplasty was noted to be 5 months. At a median follow-up of 5 months, overall success was 42%. Success after UD (n=1/10, 10%) was significantly lower than after DVIU (n=21/43, 49%; p < 0.001) with a hazard ratio of failure of 3.15 (p=0.03). DVIU was more effective after substitutional failure than after EPA (53% vs.13%, P=0.005). DVIU is more successful than UD in the management of stricture recurrence after bulbar urethroplasty. DVIU is more successful for patients with a recurrence after a substitution urethroplasty compared to after EPA, perhaps indicating a different mechanism of recurrence for EPA (ischemic) versus substitution urethroplasty (non-ischemic). Copyright © 2018 American Urological Association Education and Research, Inc. Published by Elsevier Inc. All rights reserved.

Charting epilepsy by searching for intelligence in network space with the help of evolving autonomous agents.

PubMed

Ohayon, Elan L; Kalitzin, Stiliyan; Suffczynski, Piotr; Jin, Frank Y; Tsang, Paul W; Borrett, Donald S; Burnham, W McIntyre; Kwan, Hon C

2004-01-01

The problem of demarcating neural network space is formidable. A simple fully connected recurrent network of five units (binary activations, synaptic weight resolution of 10) has 3.2 *10(26) possible initial states. The problem increases drastically with scaling. Here we consider three complementary approaches to help direct the exploration to distinguish epileptic from healthy networks. [1] First, we perform a gross mapping of the space of five-unit continuous recurrent networks using randomized weights and initial activations. The majority of weight patterns (>70%) were found to result in neural assemblies exhibiting periodic limit-cycle oscillatory behavior. [2] Next we examine the activation space of non-periodic networks demonstrating that the emergence of paroxysmal activity does not require changes in connectivity. [3] The next challenge is to focus the search of network space to identify networks with more complex dynamics. Here we rely on a major available indicator critical to clinical assessment but largely ignored by epilepsy modelers, namely: behavioral states. To this end, we connected the above network layout to an external robot in which interactive states were evolved. The first random generation showed a distribution in line with approach [1]. That is, the predominate phenotypes were fixed-point or oscillatory with seizure-like motor output. As evolution progressed the profile changed markedly. Within 20 generations the entire population was able to navigate a simple environment with all individuals exhibiting multiply-stable behaviors with no cases of default locked limit-cycle oscillatory motor behavior. The resultant population may thus afford us a view of the architectural principles demarcating healthy biological networks from the pathological. The approach has an advantage over other epilepsy modeling techniques in providing a way to clarify whether observed dynamics or suggested therapies are pointing to computational viability or dead space.

Quantitative computer tomography analysis of post-operative subdural fluid volume predicts recurrence of chronic subdural haematoma.

PubMed

Xu, Fei-Fan; Chen, Jin-Hong; Leung, Gilberto Ka Kit; Hao, Shu-Yu; Xu, Long; Hou, Zong-Gang; Mao, Xiang; Shi, Guang-Zhi; Li, Jing-Sheng; Liu, Bai-Yun

2014-01-01

Post-operative volume of subdural fluid is considered to correlate with recurrence in chronic subdural haematoma (CSDH). Information on the applications of computer-assisted volumetric analysis in patients with CSDHs is lacking. To investigate the relationship between haematoma recurrence and longitudinal changes in subdural fluid volume using CT volumetric analysis. Fifty-four patients harbouring 64 CSDHs were studied prospectively. The association between recurrence rate and CT findings were investigated. Eleven patients (20.4%) experienced post-operative recurrence. Higher pre-operative (over 120 ml) and/or pre-discharge subdural fluid volumes (over 22 ml) were significantly associated with recurrence; the probability of non-recurrence for values below these thresholds were 92.7% and 95.2%, respectively. CSDHs with larger pre-operative (over 15.1 mm) and/or residual (over 11.7 mm) widths also had significantly increased recurrence rates. Bilateral CSDHs were not found to be more likely to recur in this series. On receiver-operating characteristic curve, the areas under curve for the magnitude of changes in subdural fluid volume were greater than a single time-point measure of either width or volume of the subdural fluid cavity. Close imaging follow-up is important for CSDH patients for recurrence prediction. Using quantitative CT volumetric analysis, strong evidence was provided that changes in the residual fluid volume during the 'self-resolution' period can be used as significantly radiological predictors of recurrence.

Prioritization of Epilepsy Associated Candidate Genes by Convergent Analysis

PubMed Central

Jia, Peilin; Ewers, Jeffrey M.; Zhao, Zhongming

2011-01-01

Background Epilepsy is a severe neurological disorder affecting a large number of individuals, yet the underlying genetic risk factors for epilepsy remain unclear. Recent studies have revealed several recurrent copy number variations (CNVs) that are more likely to be associated with epilepsy. The responsible gene(s) within these regions have yet to be definitively linked to the disorder, and the implications of their interactions are not fully understood. Identification of these genes may contribute to a better pathological understanding of epilepsy, and serve to implicate novel therapeutic targets for further research. Methodology/Principal Findings In this study, we examined genes within heterozygous deletion regions identified in a recent large-scale study, encompassing a diverse spectrum of epileptic syndromes. By integrating additional protein-protein interaction data, we constructed subnetworks for these CNV-region genes and also those previously studied for epilepsy. We observed 20 genes common to both networks, primarily concentrated within a small molecular network populated by GABA receptor, BDNF/MAPK signaling, and estrogen receptor genes. From among the hundreds of genes in the initial networks, these were designated by convergent evidence for their likely association with epilepsy. Importantly, the identified molecular network was found to contain complex interrelationships, providing further insight into epilepsy's underlying pathology. We further performed pathway enrichment and crosstalk analysis and revealed a functional map which indicates the significant enrichment of closely related neurological, immune, and kinase regulatory pathways. Conclusions/Significance The convergent framework we proposed here provides a unique and powerful approach to screening and identifying promising disease genes out of typically hundreds to thousands of genes in disease-related CNV-regions. Our network and pathway analysis provides important implications for the underlying molecular mechanisms for epilepsy. The strategy can be applied for the study of other complex diseases. PMID:21390307

Prioritization of epilepsy associated candidate genes by convergent analysis.

PubMed

Jia, Peilin; Ewers, Jeffrey M; Zhao, Zhongming

2011-02-24

Epilepsy is a severe neurological disorder affecting a large number of individuals, yet the underlying genetic risk factors for epilepsy remain unclear. Recent studies have revealed several recurrent copy number variations (CNVs) that are more likely to be associated with epilepsy. The responsible gene(s) within these regions have yet to be definitively linked to the disorder, and the implications of their interactions are not fully understood. Identification of these genes may contribute to a better pathological understanding of epilepsy, and serve to implicate novel therapeutic targets for further research. In this study, we examined genes within heterozygous deletion regions identified in a recent large-scale study, encompassing a diverse spectrum of epileptic syndromes. By integrating additional protein-protein interaction data, we constructed subnetworks for these CNV-region genes and also those previously studied for epilepsy. We observed 20 genes common to both networks, primarily concentrated within a small molecular network populated by GABA receptor, BDNF/MAPK signaling, and estrogen receptor genes. From among the hundreds of genes in the initial networks, these were designated by convergent evidence for their likely association with epilepsy. Importantly, the identified molecular network was found to contain complex interrelationships, providing further insight into epilepsy's underlying pathology. We further performed pathway enrichment and crosstalk analysis and revealed a functional map which indicates the significant enrichment of closely related neurological, immune, and kinase regulatory pathways. The convergent framework we proposed here provides a unique and powerful approach to screening and identifying promising disease genes out of typically hundreds to thousands of genes in disease-related CNV-regions. Our network and pathway analysis provides important implications for the underlying molecular mechanisms for epilepsy. The strategy can be applied for the study of other complex diseases.

Extended observability of linear time-invariant systems under recurrent loss of output data

NASA Technical Reports Server (NTRS)

Luck, Rogelio; Ray, Asok; Halevi, Yoram

1989-01-01

Recurrent loss of sensor data in integrated control systems of an advanced aircraft may occur under different operating conditions that include detected frame errors and queue saturation in computer networks, and bad data suppression in signal processing. This paper presents an extension of the concept of observability based on a set of randomly selected nonconsecutive outputs in finite-dimensional, linear, time-invariant systems. Conditions for testing extended observability have been established.

Identification and Control of Aircrafts using Multiple Models and Adaptive Critics

NASA Technical Reports Server (NTRS)

Principe, Jose C.

2007-01-01

We compared two possible implementations of local linear models for control: one approach is based on a self-organizing map (SOM) to cluster the dynamics followed by a set of linear models operating at each cluster. Therefore the gating function is hard (a single local model will represent the regional dynamics). This simplifies the controller design since there is a one to one mapping between controllers and local models. The second approach uses a soft gate using a probabilistic framework based on a Gaussian Mixture Model (also called a dynamic mixture of experts). In this approach several models may be active at a given time, we can expect a smaller number of models, but the controller design is more involved, with potentially better noise rejection characteristics. Our experiments showed that the SOM provides overall best performance in high SNRs, but the performance degrades faster than with the GMM for the same noise conditions. The SOM approach required about an order of magnitude more models than the GMM, so in terms of implementation cost, the GMM is preferable. The design of the SOM is straight forward, while the design of the GMM controllers, although still reasonable, is more involved and needs more care in the selection of the parameters. Either one of these locally linear approaches outperform global nonlinear controllers based on neural networks, such as the time delay neural network (TDNN). Therefore, in essence the local model approach warrants practical implementations. In order to call the attention of the control community for this design methodology we extended successfully the multiple model approach to PID controllers (still today the most widely used control scheme in the industry), and wrote a paper on this subject. The echo state network (ESN) is a recurrent neural network with the special characteristics that only the output parameters are trained. The recurrent connections are preset according to the problem domain and are fixed. In a nutshell, the states of the reservoir of recurrent processing elements implement a projection space, where the desired response is optimally projected. This architecture trades training efficiency by a large increase in the dimension of the recurrent layer. However, the power of the recurrent neural networks can be brought to bear on practical difficult problems. Our goal was to implement an adaptive critic architecture implementing Bellman s approach to optimal control. However, we could only characterize the ESN performance as a critic in value function evaluation, which is just one of the pieces of the overall adaptive critic controller. The results were very convincing, and the simplicity of the implementation was unparalleled.

Second-order analysis of semiparametric recurrent event processes.

PubMed

Guan, Yongtao

2011-09-01

A typical recurrent event dataset consists of an often large number of recurrent event processes, each of which contains multiple event times observed from an individual during a follow-up period. Such data have become increasingly available in medical and epidemiological studies. In this article, we introduce novel procedures to conduct second-order analysis for a flexible class of semiparametric recurrent event processes. Such an analysis can provide useful information regarding the dependence structure within each recurrent event process. Specifically, we will use the proposed procedures to test whether the individual recurrent event processes are all Poisson processes and to suggest sensible alternative models for them if they are not. We apply these procedures to a well-known recurrent event dataset on chronic granulomatous disease and an epidemiological dataset on meningococcal disease cases in Merseyside, United Kingdom to illustrate their practical value. © 2011, The International Biometric Society.

Spatial recurrence analysis: A sensitive and fast detection tool in digital mammography

DOE Office of Scientific and Technical Information (OSTI.GOV)

Prado, T. L.; Galuzio, P. P.; Lopes, S. R.

Efficient diagnostics of breast cancer requires fast digital mammographic image processing. Many breast lesions, both benign and malignant, are barely visible to the untrained eye and requires accurate and reliable methods of image processing. We propose a new method of digital mammographic image analysis that meets both needs. It uses the concept of spatial recurrence as the basis of a spatial recurrence quantification analysis, which is the spatial extension of the well-known time recurrence analysis. The recurrence-based quantifiers are able to evidence breast lesions in a way as good as the best standard image processing methods available, but with amore » better control over the spurious fragments in the image.« less

Can postoperative GnRH agonist treatment prevent endometriosis recurrence? A meta-analysis.

PubMed

Zheng, Qiaomei; Mao, Hongluan; Xu, Ying; Zhao, Jing; Wei, Xuan; Liu, Peishu

2016-07-01

To investigate whether postoperative GnRH agonist (GnRH-a) treatment can prevent endometriosis recurrence. This meta-analysis searched PubMed, Embase and Cochrane Library for relevant studies published online before June 2015. Seven randomized controlled trials including 328 patients with postoperative GnRH-a treatment and 394 patients in control group were included in the meta-analysis. In the meta-analysis, the recurrence rate of GnRH-a group compared with control group was evaluated with odds ratio (OR) and its 95 % confidence interval (CI). Heterogeneity, small study effect and publication bias were, respectively, assessed using Higgins I (2), sensitivity analysis and funnel plot. Postoperative GnRH-a treatment for endometriosis (pooled OR = 0.71; 95 % CI 0.52-0.96) was superior to expectant or placebo treatment in prevention of the recurrence. The recurrence rate decreased significantly in patients who received 6 months GnRH-a treatment (pooled OR = 0.59, 95 % CI 0.38-0.90), whereas no significant difference of recurrence rate existed between patients with 3 months post-surgical GnRH-a therapy and the control group (pooled OR = 0.87, 95 % CI 0.56-1.34). No significant heterogeneity and small study effect were found in the meta-analysis. However, publication bias did existed in the present meta-analysis. Longer-term (6 months) postoperative administration of GnRH-a can decrease the recurrence risk of endometriosis, whereas 3 months duration of GnRH-a therapy makes no significant difference in preventing the recurrence of endometriosis. Therefore, instead of a 3 month therapy, the duration of the postoperative administration should be longer enough (6 months) to prevent the recurrence of endometriosis.

Using LSTM recurrent neural networks for monitoring the LHC superconducting magnets

NASA Astrophysics Data System (ADS)

Wielgosz, Maciej; Skoczeń, Andrzej; Mertik, Matej

2017-09-01

The superconducting LHC magnets are coupled with an electronic monitoring system which records and analyzes voltage time series reflecting their performance. A currently used system is based on a range of preprogrammed triggers which launches protection procedures when a misbehavior of the magnets is detected. All the procedures used in the protection equipment were designed and implemented according to known working scenarios of the system and are updated and monitored by human operators. This paper proposes a novel approach to monitoring and fault protection of the Large Hadron Collider (LHC) superconducting magnets which employs state-of-the-art Deep Learning algorithms. Consequently, the authors of the paper decided to examine the performance of LSTM recurrent neural networks for modeling of voltage time series of the magnets. In order to address this challenging task different network architectures and hyper-parameters were used to achieve the best possible performance of the solution. The regression results were measured in terms of RMSE for different number of future steps and history length taken into account for the prediction. The best result of RMSE = 0 . 00104 was obtained for a network of 128 LSTM cells within the internal layer and 16 steps history buffer.

Recurrence networks from multivariate signals for uncovering dynamic transitions of horizontal oil-water stratified flows

NASA Astrophysics Data System (ADS)

Gao, Zhong-Ke; Zhang, Xin-Wang; Jin, Ning-De; Donner, Reik V.; Marwan, Norbert; Kurths, Jürgen

2013-09-01

Characterizing the mechanism of drop formation at the interface of horizontal oil-water stratified flows is a fundamental problem eliciting a great deal of attention from different disciplines. We experimentally and theoretically investigate the formation and transition of horizontal oil-water stratified flows. We design a new multi-sector conductance sensor and measure multivariate signals from two different stratified flow patterns. Using the Adaptive Optimal Kernel Time-Frequency Representation (AOK TFR) we first characterize the flow behavior from an energy and frequency point of view. Then, we infer multivariate recurrence networks from the experimental data and investigate the cross-transitivity for each constructed network. We find that the cross-transitivity allows quantitatively uncovering the flow behavior when the stratified flow evolves from a stable state to an unstable one and recovers deeper insights into the mechanism governing the formation of droplets at the interface of stratified flows, a task that existing methods based on AOK TFR fail to work. These findings present a first step towards an improved understanding of the dynamic mechanism leading to the transition of horizontal oil-water stratified flows from a complex-network perspective.

Integrated built-in-test false and missed alarms reduction based on forward infinite impulse response & recurrent finite impulse response dynamic neural networks

NASA Astrophysics Data System (ADS)

Cui, Yiqian; Shi, Junyou; Wang, Zili

2017-11-01

Built-in tests (BITs) are widely used in mechanical systems to perform state identification, whereas the BIT false and missed alarms cause trouble to the operators or beneficiaries to make correct judgments. Artificial neural networks (ANN) are previously used for false and missed alarms identification, which has the features such as self-organizing and self-study. However, these ANN models generally do not incorporate the temporal effect of the bottom-level threshold comparison outputs and the historical temporal features are not fully considered. To improve the situation, this paper proposes a new integrated BIT design methodology by incorporating a novel type of dynamic neural networks (DNN) model. The new DNN model is termed as Forward IIR & Recurrent FIR DNN (FIRF-DNN), where its component neurons, network structures, and input/output relationships are discussed. The condition monitoring false and missed alarms reduction implementation scheme based on FIRF-DNN model is also illustrated, which is composed of three stages including model training, false and missed alarms detection, and false and missed alarms suppression. Finally, the proposed methodology is demonstrated in the application study and the experimental results are analyzed.

A recurrent neural network for classification of unevenly sampled variable stars

NASA Astrophysics Data System (ADS)

Naul, Brett; Bloom, Joshua S.; Pérez, Fernando; van der Walt, Stéfan

2018-02-01

Astronomical surveys of celestial sources produce streams of noisy time series measuring flux versus time (`light curves'). Unlike in many other physical domains, however, large (and source-specific) temporal gaps in data arise naturally due to intranight cadence choices as well as diurnal and seasonal constraints1-5. With nightly observations of millions of variable stars and transients from upcoming surveys4,6, efficient and accurate discovery and classification techniques on noisy, irregularly sampled data must be employed with minimal human-in-the-loop involvement. Machine learning for inference tasks on such data traditionally requires the laborious hand-coding of domain-specific numerical summaries of raw data (`features')7. Here, we present a novel unsupervised autoencoding recurrent neural network8 that makes explicit use of sampling times and known heteroskedastic noise properties. When trained on optical variable star catalogues, this network produces supervised classification models that rival other best-in-class approaches. We find that autoencoded features learned in one time-domain survey perform nearly as well when applied to another survey. These networks can continue to learn from new unlabelled observations and may be used in other unsupervised tasks, such as forecasting and anomaly detection.

Using Long-Short-Term-Memory Recurrent Neural Networks to Predict Aviation Engine Vibrations

NASA Astrophysics Data System (ADS)

ElSaid, AbdElRahman Ahmed

This thesis examines building viable Recurrent Neural Networks (RNN) using Long Short Term Memory (LSTM) neurons to predict aircraft engine vibrations. The different networks are trained on a large database of flight data records obtained from an airline containing flights that suffered from excessive vibration. RNNs can provide a more generalizable and robust method for prediction over analytical calculations of engine vibration, as analytical calculations must be solved iteratively based on specific empirical engine parameters, and this database contains multiple types of engines. Further, LSTM RNNs provide a "memory" of the contribution of previous time series data which can further improve predictions of future vibration values. LSTM RNNs were used over traditional RNNs, as those suffer from vanishing/exploding gradients when trained with back propagation. The study managed to predict vibration values for 1, 5, 10, and 20 seconds in the future, with 2.84% 3.3%, 5.51% and 10.19% mean absolute error, respectively. These neural networks provide a promising means for the future development of warning systems so that suitable actions can be taken before the occurrence of excess vibration to avoid unfavorable situations during flight.

Recurrence of random walks with long-range steps generated by fractional Laplacian matrices on regular networks and simple cubic lattices

NASA Astrophysics Data System (ADS)

Michelitsch, T. M.; Collet, B. A.; Riascos, A. P.; Nowakowski, A. F.; Nicolleau, F. C. G. A.

2017-12-01

We analyze a Markovian random walk strategy on undirected regular networks involving power matrix functions of the type L\\frac{α{2}} where L indicates a ‘simple’ Laplacian matrix. We refer to such walks as ‘fractional random walks’ with admissible interval 0<α ≤slant 2 . We deduce probability-generating functions (network Green’s functions) for the fractional random walk. From these analytical results we establish a generalization of Polya’s recurrence theorem for fractional random walks on d-dimensional infinite lattices: The fractional random walk is transient for dimensions d > α (recurrent for d≤slantα ) of the lattice. As a consequence, for 0<α< 1 the fractional random walk is transient for all lattice dimensions d=1, 2, .. and in the range 1≤slantα < 2 for dimensions d≥slant 2 . Finally, for α=2 , Polya’s classical recurrence theorem is recovered, namely the walk is transient only for lattice dimensions d≥slant 3 . The generalization of Polya’s recurrence theorem remains valid for the class of random walks with Lévy flight asymptotics for long-range steps. We also analyze the mean first passage probabilities, mean residence times, mean first passage times and global mean first passage times (Kemeny constant) for the fractional random walk. For an infinite 1D lattice (infinite ring) we obtain for the transient regime 0<α<1 closed form expressions for the fractional lattice Green’s function matrix containing the escape and ever passage probabilities. The ever passage probabilities (fractional lattice Green’s functions) in the transient regime fulfil Riesz potential power law decay asymptotic behavior for nodes far from the departure node. The non-locality of the fractional random walk is generated by the non-diagonality of the fractional Laplacian matrix with Lévy-type heavy tailed inverse power law decay for the probability of long-range moves. This non-local and asymptotic behavior of the fractional random walk introduces small-world properties with the emergence of Lévy flights on large (infinite) lattices.

Risk Factors for Chronic Subdural Hematoma Recurrence Identified Using Quantitative Computed Tomography Analysis of Hematoma Volume and Density.

PubMed

Stavrinou, Pantelis; Katsigiannis, Sotirios; Lee, Jong Hun; Hamisch, Christina; Krischek, Boris; Mpotsaris, Anastasios; Timmer, Marco; Goldbrunner, Roland

2017-03-01

Chronic subdural hematoma (CSDH), a common condition in elderly patients, presents a therapeutic challenge with recurrence rates of 33%. We aimed to identify specific prognostic factors for recurrence using quantitative analysis of hematoma volume and density. We retrospectively reviewed radiographic and clinical data of 227 CSDHs in 195 consecutive patients who underwent evacuation of the hematoma through a single burr hole, 2 burr holes, or a mini-craniotomy. To examine the relationship between hematoma recurrence and various clinical, radiologic, and surgical factors, we used quantitative image-based analysis to measure the hematoma and trapped air volumes and the hematoma densities. Recurrence of CSDH occurred in 35 patients (17.9%). Multivariate logistic regression analysis revealed that the percentage of hematoma drained and postoperative CSDH density were independent risk factors for recurrence. All 3 evacuation methods were equally effective in draining the hematoma (71.7% vs. 73.7% vs. 71.9%) without observable differences in postoperative air volume captured in the subdural space. Quantitative image analysis provided evidence that percentage of hematoma drained and postoperative CSDH density are independent prognostic factors for subdural hematoma recurrence. Copyright © 2016 Elsevier Inc. All rights reserved.

AAAIC '88 - Aerospace Applications of Artificial Intelligence; Proceedings of the Fourth Annual Conference, Dayton, OH, Oct. 25-27, 1988. Volumes 1 2

DOE Office of Scientific and Technical Information (OSTI.GOV)

Johnson, J.R.; Netrologic, Inc., San Diego, CA)

1988-01-01

Topics presented include integrating neural networks and expert systems, neural networks and signal processing, machine learning, cognition and avionics applications, artificial intelligence and man-machine interface issues, real time expert systems, artificial intelligence, and engineering applications. Also considered are advanced problem solving techniques, combinational optimization for scheduling and resource control, data fusion/sensor fusion, back propagation with momentum, shared weights and recurrency, automatic target recognition, cybernetics, optical neural networks.

A Novel Connectionist Network for Solving Long Time-Lag Prediction Tasks

NASA Astrophysics Data System (ADS)

Johnson, Keith; MacNish, Cara

Traditional Recurrent Neural Networks (RNNs) perform poorly on learning tasks involving long time-lag dependencies. More recent approaches such as LSTM and its variants significantly improve on RNNs ability to learn this type of problem. We present an alternative approach to encoding temporal dependencies that associates temporal features with nodes rather than state values, where the nodes explicitly encode dependencies over variable time delays. We show promising results comparing the network's performance to LSTM variants on an extended Reber grammar task.

Heterogeneous recurrence monitoring and control of nonlinear stochastic processes

DOE Office of Scientific and Technical Information (OSTI.GOV)

Yang, Hui, E-mail: huiyang@usf.edu; Chen, Yun

Recurrence is one of the most common phenomena in natural and engineering systems. Process monitoring of dynamic transitions in nonlinear and nonstationary systems is more concerned with aperiodic recurrences and recurrence variations. However, little has been done to investigate the heterogeneous recurrence variations and link with the objectives of process monitoring and anomaly detection. Notably, nonlinear recurrence methodologies are based on homogeneous recurrences, which treat all recurrence states in the same way as black dots, and non-recurrence is white in recurrence plots. Heterogeneous recurrences are more concerned about the variations of recurrence states in terms of state properties (e.g., valuesmore » and relative locations) and the evolving dynamics (e.g., sequential state transitions). This paper presents a novel approach of heterogeneous recurrence analysis that utilizes a new fractal representation to delineate heterogeneous recurrence states in multiple scales, including the recurrences of both single states and multi-state sequences. Further, we developed a new set of heterogeneous recurrence quantifiers that are extracted from fractal representation in the transformed space. To that end, we integrated multivariate statistical control charts with heterogeneous recurrence analysis to simultaneously monitor two or more related quantifiers. Experimental results on nonlinear stochastic processes show that the proposed approach not only captures heterogeneous recurrence patterns in the fractal representation but also effectively monitors the changes in the dynamics of a complex system.« less

Computations in the deep vs superficial layers of the cerebral cortex.

PubMed

Rolls, Edmund T; Mills, W Patrick C

2017-11-01

A fundamental question is how the cerebral neocortex operates functionally, computationally. The cerebral neocortex with its superficial and deep layers and highly developed recurrent collateral systems that provide a basis for memory-related processing might perform somewhat different computations in the superficial and deep layers. Here we take into account the quantitative connectivity within and between laminae. Using integrate-and-fire neuronal network simulations that incorporate this connectivity, we first show that attractor networks implemented in the deep layers that are activated by the superficial layers could be partly independent in that the deep layers might have a different time course, which might because of adaptation be more transient and useful for outputs from the neocortex. In contrast the superficial layers could implement more prolonged firing, useful for slow learning and for short-term memory. Second, we show that a different type of computation could in principle be performed in the superficial and deep layers, by showing that the superficial layers could operate as a discrete attractor network useful for categorisation and feeding information forward up a cortical hierarchy, whereas the deep layers could operate as a continuous attractor network useful for providing a spatially and temporally smooth output to output systems in the brain. A key advance is that we draw attention to the functions of the recurrent collateral connections between cortical pyramidal cells, often omitted in canonical models of the neocortex, and address principles of operation of the neocortex by which the superficial and deep layers might be specialized for different types of attractor-related memory functions implemented by the recurrent collaterals. Copyright © 2017 Elsevier Inc. All rights reserved.

Analysis of fMRI data using noise-diffusion network models: a new covariance-coding perspective.

PubMed

Gilson, Matthieu

2018-04-01

Since the middle of the 1990s, studies of resting-state fMRI/BOLD data have explored the correlation patterns of activity across the whole brain, which is referred to as functional connectivity (FC). Among the many methods that have been developed to interpret FC, a recently proposed model-based approach describes the propagation of fluctuating BOLD activity within the recurrently connected brain network by inferring the effective connectivity (EC). In this model, EC quantifies the strengths of directional interactions between brain regions, viewed from the proxy of BOLD activity. In addition, the tuning procedure for the model provides estimates for the local variability (input variances) to explain how the observed FC is generated. Generalizing, the network dynamics can be studied in the context of an input-output mapping-determined by EC-for the second-order statistics of fluctuating nodal activities. The present paper focuses on the following detection paradigm: observing output covariances, how discriminative is the (estimated) network model with respect to various input covariance patterns? An application with the model fitted to experimental fMRI data-movie viewing versus resting state-illustrates that changes in local variability and changes in brain coordination go hand in hand.

Artificial Epigenetic Networks: Automatic Decomposition of Dynamical Control Tasks Using Topological Self-Modification.

PubMed

Turner, Alexander P; Caves, Leo S D; Stepney, Susan; Tyrrell, Andy M; Lones, Michael A

2017-01-01

This paper describes the artificial epigenetic network, a recurrent connectionist architecture that is able to dynamically modify its topology in order to automatically decompose and solve dynamical problems. The approach is motivated by the behavior of gene regulatory networks, particularly the epigenetic process of chromatin remodeling that leads to topological change and which underlies the differentiation of cells within complex biological organisms. We expected this approach to be useful in situations where there is a need to switch between different dynamical behaviors, and do so in a sensitive and robust manner in the absence of a priori information about problem structure. This hypothesis was tested using a series of dynamical control tasks, each requiring solutions that could express different dynamical behaviors at different stages within the task. In each case, the addition of topological self-modification was shown to improve the performance and robustness of controllers. We believe this is due to the ability of topological changes to stabilize attractors, promoting stability within a dynamical regime while allowing rapid switching between different regimes. Post hoc analysis of the controllers also demonstrated how the partitioning of the networks could provide new insights into problem structure.

Occult lymph node metastasis and risk of regional recurrence in papillary thyroid cancer after bilateral prophylactic central neck dissection: A multi-institutional study.

PubMed

Lee, Young Chan; Na, Se Young; Park, Gi Cheol; Han, Ju Hyun; Kim, Seung Woo; Eun, Young Gyu

2017-02-01

The impact of occult lymph node metastasis on regional recurrence after prophylactic central neck dissection for preoperative, nodal-negative papillary thyroid cancer is controversial. We investigated risk factors for regional lymph node recurrence in papillary thyroid cancer patients who underwent total thyroidectomy and bilateral prophylactic central neck dissection. Analysis was according to clinicopathologic characteristics and occult lymph node metastasis patterns. This multicenter study enrolled 211 consecutive patients who underwent total thyroidectomy with bilateral prophylactic central neck dissection for papillary thyroid cancer without evidence of central lymph node metastasis on preoperative imaging. Clinicopathologic features and central lymph node metastasis patterns were analyzed for predicting regional recurrence. Multivariate Cox regression analysis was used to identify independent factors for recurrence. Median follow-up time was 43 months (24-95 months). Ten patients (4.7%) showed regional lymph node recurrence. The estimated 5-year, regional recurrence-free survival was 95.2%. Tumor size ≥1 cm, central lymph node metastasis, lymph node ratio, and prelaryngeal lymph node metastasis were associated with regional recurrence in univariate analysis (P < .05). In multivariate analysis, a lymph node ratio ≥ 0.26 was a significant risk factor for regional lymph node recurrence (odds ratio = 11.63, P = .003). Lymph node ratio ≥ 0.26 was an independent predictor of worse recurrence-free survival on Cox regression analysis (hazard ratio = 11.49, P = .002). Although no significant association was observed between the presence of occult lymph node metastasis and regional recurrence, lymph node ratio ≥ 0.26 was an independent predictor of regional lymph node recurrence in papillary thyroid cancer patients who underwent total thyroidectomy and bilateral prophylactic central neck dissection. Copyright © 2016 Elsevier Inc. All rights reserved.

Deep learning for pharmacovigilance: recurrent neural network architectures for labeling adverse drug reactions in Twitter posts.

PubMed

Cocos, Anne; Fiks, Alexander G; Masino, Aaron J

2017-07-01

Social media is an important pharmacovigilance data source for adverse drug reaction (ADR) identification. Human review of social media data is infeasible due to data quantity, thus natural language processing techniques are necessary. Social media includes informal vocabulary and irregular grammar, which challenge natural language processing methods. Our objective is to develop a scalable, deep-learning approach that exceeds state-of-the-art ADR detection performance in social media. We developed a recurrent neural network (RNN) model that labels words in an input sequence with ADR membership tags. The only input features are word-embedding vectors, which can be formed through task-independent pretraining or during ADR detection training. Our best-performing RNN model used pretrained word embeddings created from a large, non-domain-specific Twitter dataset. It achieved an approximate match F-measure of 0.755 for ADR identification on the dataset, compared to 0.631 for a baseline lexicon system and 0.65 for the state-of-the-art conditional random field model. Feature analysis indicated that semantic information in pretrained word embeddings boosted sensitivity and, combined with contextual awareness captured in the RNN, precision. Our model required no task-specific feature engineering, suggesting generalizability to additional sequence-labeling tasks. Learning curve analysis showed that our model reached optimal performance with fewer training examples than the other models. ADR detection performance in social media is significantly improved by using a contextually aware model and word embeddings formed from large, unlabeled datasets. The approach reduces manual data-labeling requirements and is scalable to large social media datasets. © The Author 2017. Published by Oxford University Press on behalf of the American Medical Informatics Association. All rights reserved. For Permissions, please email: journals.permissions@oup.com

Antithrombotic Treatment for Recurrent Miscarriage

PubMed Central

Zhang, Tianyi; Ye, Xiaofei; Zhu, Tiantian; XIAO, Xiang; Liu, Yuzhou; Wei, Xin; Liu, Yu; Wu, Cheng; Guan, Rui; Li, Xiao; Guo, Xiaojing; Hu, Huili; He, Jia

2015-01-01

Abstract Combined use of heparin and aspirin is frequently prescribed for treatment of recurrent miscarriage (RM) in patients with antiphospholipid syndrome (APS), or in those without apparent cause of RM other than thrombophilia; however, this strategy is largely based on expert opinion and has not been well studied. The option for the use of different antithrombotic therapies to improve live birth remains unclear. In this network meta-analysis, we incorporated direct and indirect evidence to evaluate effects of different antithrombotic treatments on prevention of pregnancy losses. We searched PubMed and Embase for randomized clinical trials comparing effects of at least 2 antithrombotic treatments on live birth in RM patients published from 1965 through the early of May 2015. Potential risk bias of eligible trials was evaluated according to the Cochrane Collaboration guidelines. Bayesian network meta-analysis was used to estimate relative effects on live birth. A total of 19 trials involving 2391 RM patients with or without thrombophilia and 543 with APS were included. No beneficial effect of antithrombotic treatment was observed either in RM patients with or without thrombophilia or in patients with APS; however, for patients with or without thrombophilia, low molecular weight heparin therapy had the greatest probability (61.48%) of being the best option in terms of live birth; for patients with APS, unfractionated heparin plus aspirin was the superior treatment for RM with the highest possibility (75.15%) of being top 2 places for reducing pregnancy losses. Aspirin was inferior in both groups. Our results do not support the use of combined low molecular weight heparin and aspirin for RM treatment, and suggested aspirin may have negative effects for lowering the risk of pregnancy loss. PMID:26559249

A spatially explicit metapopulation model and cattle trade analysis suggests key determinants for the recurrent circulation of rift valley Fever virus in a pilot area of madagascar highlands.

PubMed

Nicolas, Gaëlle; Chevalier, Véronique; Tantely, Luciano Michaël; Fontenille, Didier; Durand, Benoît

2014-12-01

Rift Valley fever (RVF) is a vector-borne zoonotic disease that causes high morbidity and mortality in ruminants. In 2008-2009, a RVF outbreak affected the whole Madagascar island, including the Anjozorobe district located in Madagascar highlands. An entomological survey showed the absence of Aedes among the potential RVF virus (RVFV) vector species identified in this area, and an overall low abundance of mosquitoes due to unfavorable climatic conditions during winter. No serological nor virological sign of infection was observed in wild terrestrial mammals of the area, suggesting an absence of wild RVF virus (RVFV) reservoir. However, a three years serological and virological follow-up in cattle showed a recurrent RVFV circulation. The objective of this study was to understand the key determinants of this unexpected recurrent transmission. To achieve this goal, a spatial deterministic discrete-time metapopulation model combined with cattle trade network was designed and parameterized to reproduce the local conditions using observational data collected in the area. Three scenarios that could explain the RVFV recurrent circulation in the area were analyzed: (i) RVFV overwintering thanks to a direct transmission between cattle when viraemic cows calve, vectors being absent during the winter, (ii) a low level vector-based circulation during winter thanks to a residual vector population, without direct transmission between cattle, (iii) combination of both above mentioned mechanisms. Multi-model inference methods resulted in a model incorporating both a low level RVFV winter vector-borne transmission and a direct transmission between animals when viraemic cows calve. Predictions satisfactorily reproduced field observations, 84% of cattle infections being attributed to vector-borne transmission, and 16% to direct transmission. These results appeared robust according to the sensitivity analysis. Interweaving between agricultural works in rice fields, seasonality of vector proliferation, and cattle exchange practices could be a key element for understanding RVFV circulation in this area of Madagascar highlands.

Neural network decoder for quantum error correcting codes

NASA Astrophysics Data System (ADS)

Krastanov, Stefan; Jiang, Liang

Artificial neural networks form a family of extremely powerful - albeit still poorly understood - tools used in anything from image and sound recognition through text generation to, in our case, decoding. We present a straightforward Recurrent Neural Network architecture capable of deducing the correcting procedure for a quantum error-correcting code from a set of repeated stabilizer measurements. We discuss the fault-tolerance of our scheme and the cost of training the neural network for a system of a realistic size. Such decoders are especially interesting when applied to codes, like the quantum LDPC codes, that lack known efficient decoding schemes.

Analysis of the streamflow-gaging station network in Ohio for effectiveness in providing regional streamflow information

USGS Publications Warehouse

Straub, D.E.

1998-01-01

The streamflow-gaging station network in Ohio was evaluated for its effectiveness in providing regional streamflow information. The analysis involved application of the principles of generalized least squares regression between streamflow and climatic and basin characteristics. Regression equations were developed for three flow characteristics: (1) the instantaneous peak flow with a 100-year recurrence interval (P100), (2) the mean annual flow (Qa), and (3) the 7-day, 10-year low flow (7Q10). All active and discontinued gaging stations with 5 or more years of unregulated-streamflow data with respect to each flow characteristic were used to develop the regression equations. The gaging-station network was evaluated for the current (1996) condition of the network and estimated conditions of various network strategies if an additional 5 and 20 years of streamflow data were collected. Any active or discontinued gaging station with (1) less than 5 years of unregulated-streamflow record, (2) previously defined basin and climatic characteristics, and (3) the potential for collection of more unregulated-streamflow record were included in the network strategies involving the additional 5 and 20 years of data. The network analysis involved use of the regression equations, in combination with location, period of record, and cost of operation, to determine the contribution of the data for each gaging station to regional streamflow information. The contribution of each gaging station was based on a cost-weighted reduction of the mean square error (average sampling-error variance) associated with each regional estimating equation. All gaging stations included in the network analysis were then ranked according to their contribution to the regional information for each flow characteristic. The predictive ability of the regression equations developed from the gaging station network could be improved for all three flow characteristics with the collection of additional streamflow data. The addition of new gaging stations to the network would result in an even greater improvement of the accuracy of the regional regression equations. Typically, continued data collection at stations with unregulated streamflow for all flow conditions that had less than 11 years of record with drainage areas smaller than 200 square miles contributed the largest cost-weighted reduction to the average sampling-error variance of the regional estimating equations. The results of the network analyses can be used to prioritize the continued operation of active gaging stations or the reactivation of discontinued gaging stations if the objective is to maximize the regional information content in the streamflow-gaging station network.

Personality and social support as predictors of first and recurrent episodes of depression.

PubMed

Noteboom, Annemieke; Beekman, Aartjan T F; Vogelzangs, Nicole; Penninx, Brenda W J H

2016-01-15

Depression is a prevalent psychiatric disorder with high personal and public health consequences, partly due to a high risk of recurrence. This longitudinal study examines personality traits, structural and subjective social support dimensions as predictors of first and recurrent episodes of depression in initially non-depressed subjects. Data were obtained from the Netherlands Study of Depression and Anxiety (NESDA). 1085 respondents without a current depression or anxiety diagnosis were included. 437 respondents had a prior history of depression, 648 did not. Personality dimensions were measured with the NEO-FFI, network size, partner-status, negative and positive emotional support were measured with the Close Person Questionnaire. Logistic regression analyses (unadjusted and adjusted for clinical variables and sociodemographic variables) examined whether these psychosocial variables predict a new episode of depression at two year follow up and whether this differed among persons with or without a history of depression. In the unadjusted analyses high extraversion (OR:.93, 95% CI (.91-.96), P<.001), agreeableness (OR:.94, 95% CI (.90-.97), P<.001), conscientiousness (OR:.93, 95% CI (.90-.96), P<.001) and a larger network size (OR:.76, 95% CI (.64-.90), P=.001) significantly reduced the risk of a new episode of depression. Only neuroticism predicted a new episode of depression in both the unadjusted (OR:1.13, 95% CI (1.10-1.15), P<.001) and adjusted analyses (OR:1.06, 95% CI (1.03-1.10), P<.001). None of the predictors predicted first or recurrent episodes of depression differently. we used a relatively short follow up period and broad personality dimensions. Neuroticism seems to predict both first and recurrent episodes of depression and may be suitable for screening for preventive interventions. Copyright © 2015 Elsevier B.V. All rights reserved.

Structural network heterogeneities and network dynamics: a possible dynamical mechanism for hippocampal memory reactivation.

NASA Astrophysics Data System (ADS)

Jablonski, Piotr; Poe, Gina; Zochowski, Michal

2007-03-01

The hippocampus has the capacity for reactivating recently acquired memories and it is hypothesized that one of the functions of sleep reactivation is the facilitation of consolidation of novel memory traces. The dynamic and network processes underlying such a reactivation remain, however, unknown. We show that such a reactivation characterized by local, self-sustained activity of a network region may be an inherent property of the recurrent excitatory-inhibitory network with a heterogeneous structure. The entry into the reactivation phase is mediated through a physiologically feasible regulation of global excitability and external input sources, while the reactivated component of the network is formed through induced network heterogeneities during learning. We show that structural changes needed for robust reactivation of a given network region are well within known physiological parameters.

Structural network heterogeneities and network dynamics: A possible dynamical mechanism for hippocampal memory reactivation

NASA Astrophysics Data System (ADS)

Jablonski, Piotr; Poe, Gina R.; Zochowski, Michal

2007-01-01

The hippocampus has the capacity for reactivating recently acquired memories and it is hypothesized that one of the functions of sleep reactivation is the facilitation of consolidation of novel memory traces. The dynamic and network processes underlying such a reactivation remain, however, unknown. We show that such a reactivation characterized by local, self-sustained activity of a network region may be an inherent property of the recurrent excitatory-inhibitory network with a heterogeneous structure. The entry into the reactivation phase is mediated through a physiologically feasible regulation of global excitability and external input sources, while the reactivated component of the network is formed through induced network heterogeneities during learning. We show that structural changes needed for robust reactivation of a given network region are well within known physiological parameters.

H∞ state estimation for discrete-time memristive recurrent neural networks with stochastic time-delays

NASA Astrophysics Data System (ADS)

Liu, Hongjian; Wang, Zidong; Shen, Bo; Alsaadi, Fuad E.

2016-07-01

This paper deals with the robust H∞ state estimation problem for a class of memristive recurrent neural networks with stochastic time-delays. The stochastic time-delays under consideration are governed by a Bernoulli-distributed stochastic sequence. The purpose of the addressed problem is to design the robust state estimator such that the dynamics of the estimation error is exponentially stable in the mean square, and the prescribed ? performance constraint is met. By utilizing the difference inclusion theory and choosing a proper Lyapunov-Krasovskii functional, the existence condition of the desired estimator is derived. Based on it, the explicit expression of the estimator gain is given in terms of the solution to a linear matrix inequality. Finally, a numerical example is employed to demonstrate the effectiveness and applicability of the proposed estimation approach.

Auto-Associative Recurrent Neural Networks and Long Term Dependencies in Novelty Detection for Audio Surveillance Applications

NASA Astrophysics Data System (ADS)

Rossi, A.; Montefoschi, F.; Rizzo, A.; Diligenti, M.; Festucci, C.

2017-10-01

Machine Learning applied to Automatic Audio Surveillance has been attracting increasing attention in recent years. In spite of several investigations based on a large number of different approaches, little attention had been paid to the environmental temporal evolution of the input signal. In this work, we propose an exploration in this direction comparing the temporal correlations extracted at the feature level with the one learned by a representational structure. To this aim we analysed the prediction performances of a Recurrent Neural Network architecture varying the length of the processed input sequence and the size of the time window used in the feature extraction. Results corroborated the hypothesis that sequential models work better when dealing with data characterized by temporal order. However, so far the optimization of the temporal dimension remains an open issue.

On the correlation between reservoir metrics and performance for time series classification under the influence of synaptic plasticity.

PubMed

Chrol-Cannon, Joseph; Jin, Yaochu

2014-01-01

Reservoir computing provides a simpler paradigm of training recurrent networks by initialising and adapting the recurrent connections separately to a supervised linear readout. This creates a problem, though. As the recurrent weights and topology are now separated from adapting to the task, there is a burden on the reservoir designer to construct an effective network that happens to produce state vectors that can be mapped linearly into the desired outputs. Guidance in forming a reservoir can be through the use of some established metrics which link a number of theoretical properties of the reservoir computing paradigm to quantitative measures that can be used to evaluate the effectiveness of a given design. We provide a comprehensive empirical study of four metrics; class separation, kernel quality, Lyapunov's exponent and spectral radius. These metrics are each compared over a number of repeated runs, for different reservoir computing set-ups that include three types of network topology and three mechanisms of weight adaptation through synaptic plasticity. Each combination of these methods is tested on two time-series classification problems. We find that the two metrics that correlate most strongly with the classification performance are Lyapunov's exponent and kernel quality. It is also evident in the comparisons that these two metrics both measure a similar property of the reservoir dynamics. We also find that class separation and spectral radius are both less reliable and less effective in predicting performance.

A Markov model for the temporal dynamics of balanced random networks of finite size

PubMed Central

Lagzi, Fereshteh; Rotter, Stefan

2014-01-01

The balanced state of recurrent networks of excitatory and inhibitory spiking neurons is characterized by fluctuations of population activity about an attractive fixed point. Numerical simulations show that these dynamics are essentially nonlinear, and the intrinsic noise (self-generated fluctuations) in networks of finite size is state-dependent. Therefore, stochastic differential equations with additive noise of fixed amplitude cannot provide an adequate description of the stochastic dynamics. The noise model should, rather, result from a self-consistent description of the network dynamics. Here, we consider a two-state Markovian neuron model, where spikes correspond to transitions from the active state to the refractory state. Excitatory and inhibitory input to this neuron affects the transition rates between the two states. The corresponding nonlinear dependencies can be identified directly from numerical simulations of networks of leaky integrate-and-fire neurons, discretized at a time resolution in the sub-millisecond range. Deterministic mean-field equations, and a noise component that depends on the dynamic state of the network, are obtained from this model. The resulting stochastic model reflects the behavior observed in numerical simulations quite well, irrespective of the size of the network. In particular, a strong temporal correlation between the two populations, a hallmark of the balanced state in random recurrent networks, are well represented by our model. Numerical simulations of such networks show that a log-normal distribution of short-term spike counts is a property of balanced random networks with fixed in-degree that has not been considered before, and our model shares this statistical property. Furthermore, the reconstruction of the flow from simulated time series suggests that the mean-field dynamics of finite-size networks are essentially of Wilson-Cowan type. We expect that this novel nonlinear stochastic model of the interaction between neuronal populations also opens new doors to analyze the joint dynamics of multiple interacting networks. PMID:25520644

Decorrelation of Neural-Network Activity by Inhibitory Feedback

PubMed Central

Einevoll, Gaute T.; Diesmann, Markus

2012-01-01

Correlations in spike-train ensembles can seriously impair the encoding of information by their spatio-temporal structure. An inevitable source of correlation in finite neural networks is common presynaptic input to pairs of neurons. Recent studies demonstrate that spike correlations in recurrent neural networks are considerably smaller than expected based on the amount of shared presynaptic input. Here, we explain this observation by means of a linear network model and simulations of networks of leaky integrate-and-fire neurons. We show that inhibitory feedback efficiently suppresses pairwise correlations and, hence, population-rate fluctuations, thereby assigning inhibitory neurons the new role of active decorrelation. We quantify this decorrelation by comparing the responses of the intact recurrent network (feedback system) and systems where the statistics of the feedback channel is perturbed (feedforward system). Manipulations of the feedback statistics can lead to a significant increase in the power and coherence of the population response. In particular, neglecting correlations within the ensemble of feedback channels or between the external stimulus and the feedback amplifies population-rate fluctuations by orders of magnitude. The fluctuation suppression in homogeneous inhibitory networks is explained by a negative feedback loop in the one-dimensional dynamics of the compound activity. Similarly, a change of coordinates exposes an effective negative feedback loop in the compound dynamics of stable excitatory-inhibitory networks. The suppression of input correlations in finite networks is explained by the population averaged correlations in the linear network model: In purely inhibitory networks, shared-input correlations are canceled by negative spike-train correlations. In excitatory-inhibitory networks, spike-train correlations are typically positive. Here, the suppression of input correlations is not a result of the mere existence of correlations between excitatory (E) and inhibitory (I) neurons, but a consequence of a particular structure of correlations among the three possible pairings (EE, EI, II). PMID:23133368

Using Elman recurrent neural networks with conjugate gradient algorithm in determining the anesthetic the amount of anesthetic medicine to be applied.

PubMed

Güntürkün, Rüştü

2010-08-01

In this study, Elman recurrent neural networks have been defined by using conjugate gradient algorithm in order to determine the depth of anesthesia in the continuation stage of the anesthesia and to estimate the amount of medicine to be applied at that moment. The feed forward neural networks are also used for comparison. The conjugate gradient algorithm is compared with back propagation (BP) for training of the neural Networks. The applied artificial neural network is composed of three layers, namely the input layer, the hidden layer and the output layer. The nonlinear activation function sigmoid (sigmoid function) has been used in the hidden layer and the output layer. EEG data has been recorded with Nihon Kohden 9200 brand 22-channel EEG device. The international 8-channel bipolar 10-20 montage system (8 TB-b system) has been used in assembling the recording electrodes. EEG data have been recorded by being sampled once in every 2 milliseconds. The artificial neural network has been designed so as to have 60 neurons in the input layer, 30 neurons in the hidden layer and 1 neuron in the output layer. The values of the power spectral density (PSD) of 10-second EEG segments which correspond to the 1-50 Hz frequency range; the ratio of the total power of PSD values of the EEG segment at that moment in the same range to the total of PSD values of EEG segment taken prior to the anesthesia.

A loop-based neural architecture for structured behavior encoding and decoding.

PubMed

Gisiger, Thomas; Boukadoum, Mounir

2018-02-01

We present a new type of artificial neural network that generalizes on anatomical and dynamical aspects of the mammal brain. Its main novelty lies in its topological structure which is built as an array of interacting elementary motifs shaped like loops. These loops come in various types and can implement functions such as gating, inhibitory or executive control, or encoding of task elements to name a few. Each loop features two sets of neurons and a control region, linked together by non-recurrent projections. The two neural sets do the bulk of the loop's computations while the control unit specifies the timing and the conditions under which the computations implemented by the loop are to be performed. By functionally linking many such loops together, a neural network is obtained that may perform complex cognitive computations. To demonstrate the potential offered by such a system, we present two neural network simulations. The first illustrates the structure and dynamics of a single loop implementing a simple gating mechanism. The second simulation shows how connecting four loops in series can produce neural activity patterns that are sufficient to pass a simplified delayed-response task. We also show that this network reproduces electrophysiological measurements gathered in various regions of the brain of monkeys performing similar tasks. We also demonstrate connections between this type of neural network and recurrent or long short-term memory network models, and suggest ways to generalize them for future artificial intelligence research. Copyright © 2017 Elsevier Ltd. All rights reserved.

A modular architecture for transparent computation in recurrent neural networks.

PubMed

Carmantini, Giovanni S; Beim Graben, Peter; Desroches, Mathieu; Rodrigues, Serafim

2017-01-01

Computation is classically studied in terms of automata, formal languages and algorithms; yet, the relation between neural dynamics and symbolic representations and operations is still unclear in traditional eliminative connectionism. Therefore, we suggest a unique perspective on this central issue, to which we would like to refer as transparent connectionism, by proposing accounts of how symbolic computation can be implemented in neural substrates. In this study we first introduce a new model of dynamics on a symbolic space, the versatile shift, showing that it supports the real-time simulation of a range of automata. We then show that the Gödelization of versatile shifts defines nonlinear dynamical automata, dynamical systems evolving on a vectorial space. Finally, we present a mapping between nonlinear dynamical automata and recurrent artificial neural networks. The mapping defines an architecture characterized by its granular modularity, where data, symbolic operations and their control are not only distinguishable in activation space, but also spatially localizable in the network itself, while maintaining a distributed encoding of symbolic representations. The resulting networks simulate automata in real-time and are programmed directly, in the absence of network training. To discuss the unique characteristics of the architecture and their consequences, we present two examples: (i) the design of a Central Pattern Generator from a finite-state locomotive controller, and (ii) the creation of a network simulating a system of interactive automata that supports the parsing of garden-path sentences as investigated in psycholinguistics experiments. Copyright © 2016 Elsevier Ltd. All rights reserved.

Deep Recurrent Neural Networks for Supernovae Classification

NASA Astrophysics Data System (ADS)

Charnock, Tom; Moss, Adam

2017-03-01

We apply deep recurrent neural networks, which are capable of learning complex sequential information, to classify supernovae (code available at https://github.com/adammoss/supernovae). The observational time and filter fluxes are used as inputs to the network, but since the inputs are agnostic, additional data such as host galaxy information can also be included. Using the Supernovae Photometric Classification Challenge (SPCC) data, we find that deep networks are capable of learning about light curves, however the performance of the network is highly sensitive to the amount of training data. For a training size of 50% of the representational SPCC data set (around 104 supernovae) we obtain a type-Ia versus non-type-Ia classification accuracy of 94.7%, an area under the Receiver Operating Characteristic curve AUC of 0.986 and an SPCC figure-of-merit F 1 = 0.64. When using only the data for the early-epoch challenge defined by the SPCC, we achieve a classification accuracy of 93.1%, AUC of 0.977, and F 1 = 0.58, results almost as good as with the whole light curve. By employing bidirectional neural networks, we can acquire impressive classification results between supernovae types I, II and III at an accuracy of 90.4% and AUC of 0.974. We also apply a pre-trained model to obtain classification probabilities as a function of time and show that it can give early indications of supernovae type. Our method is competitive with existing algorithms and has applications for future large-scale photometric surveys.

Quaternion-valued echo state networks.

PubMed

Xia, Yili; Jahanchahi, Cyrus; Mandic, Danilo P

2015-04-01

Quaternion-valued echo state networks (QESNs) are introduced to cater for 3-D and 4-D processes, such as those observed in the context of renewable energy (3-D wind modeling) and human centered computing (3-D inertial body sensors). The introduction of QESNs is made possible by the recent emergence of quaternion nonlinear activation functions with local analytic properties, required by nonlinear gradient descent training algorithms. To make QENSs second-order optimal for the generality of quaternion signals (both circular and noncircular), we employ augmented quaternion statistics to introduce widely linear QESNs. To that end, the standard widely linear model is modified so as to suit the properties of dynamical reservoir, typically realized by recurrent neural networks. This allows for a full exploitation of second-order information in the data, contained both in the covariance and pseudocovariances, and a rigorous account of second-order noncircularity (improperness), and the corresponding power mismatch and coupling between the data components. Simulations in the prediction setting on both benchmark circular and noncircular signals and on noncircular real-world 3-D body motion data support the analysis.

A Two-Factor Model of Relapse/Recurrence Vulnerability in Unipolar Depression

PubMed Central

Farb, Norman A. S.; Irving, Julie A.; Anderson, Adam K.; Segal, Zindel V.

2015-01-01

The substantial health burden associated with Major Depressive Disorder is a product of both its high prevalence and the significant risk of relapse, recurrence and chronicity. Establishing recurrence vulnerability factors (VFs) could improve the long-term management of MDD by identifying the need for further intervention in seemingly recovered patients. We present a model of sensitization in depression vulnerability, with an emphasis on the integration of behavioral and neural systems accounts. Evidence suggests that VFs fall into two categories: dysphoric attention and dysphoric elaboration. Dysphoric attention is driven by fixation on negative life events, and is characterized behaviorally by reduced executive control, and neurally by elevated activity in the brain’s salience network. Dysphoric elaboration is driven by rumination that promotes over-general self and contextual appraisals, and is characterized behaviorally by dysfunctional attitudes, and neurally by elevated connectivity within normally-distinct prefrontal brain networks. While, at present, few prospective VF studies exist from which to catalogue a definitive neurobehavioral account, extant data support the value of the proposed two-factor model. Measuring the continued presence of these two VFs during recovery may more accurately identify remitted patients who would benefit from targeted prophylactic intervention. PMID:25688431

Deep Recurrent Neural Networks for seizure detection and early seizure detection systems

DOE Office of Scientific and Technical Information (OSTI.GOV)

Talathi, S. S.

Epilepsy is common neurological diseases, affecting about 0.6-0.8 % of world population. Epileptic patients suffer from chronic unprovoked seizures, which can result in broad spectrum of debilitating medical and social consequences. Since seizures, in general, occur infrequently and are unpredictable, automated seizure detection systems are recommended to screen for seizures during long-term electroencephalogram (EEG) recordings. In addition, systems for early seizure detection can lead to the development of new types of intervention systems that are designed to control or shorten the duration of seizure events. In this article, we investigate the utility of recurrent neural networks (RNNs) in designing seizuremore » detection and early seizure detection systems. We propose a deep learning framework via the use of Gated Recurrent Unit (GRU) RNNs for seizure detection. We use publicly available data in order to evaluate our method and demonstrate very promising evaluation results with overall accuracy close to 100 %. We also systematically investigate the application of our method for early seizure warning systems. Our method can detect about 98% of seizure events within the first 5 seconds of the overall epileptic seizure duration.« less

Anti-correlations in the degree distribution increase stimulus detection performance in noisy spiking neural networks.

PubMed

Martens, Marijn B; Houweling, Arthur R; E Tiesinga, Paul H

2017-02-01

Neuronal circuits in the rodent barrel cortex are characterized by stable low firing rates. However, recent experiments show that short spike trains elicited by electrical stimulation in single neurons can induce behavioral responses. Hence, the underlying neural networks provide stability against internal fluctuations in the firing rate, while simultaneously making the circuits sensitive to small external perturbations. Here we studied whether stability and sensitivity are affected by the connectivity structure in recurrently connected spiking networks. We found that anti-correlation between the number of afferent (in-degree) and efferent (out-degree) synaptic connections of neurons increases stability against pathological bursting, relative to networks where the degrees were either positively correlated or uncorrelated. In the stable network state, stimulation of a few cells could lead to a detectable change in the firing rate. To quantify the ability of networks to detect the stimulation, we used a receiver operating characteristic (ROC) analysis. For a given level of background noise, networks with anti-correlated degrees displayed the lowest false positive rates, and consequently had the highest stimulus detection performance. We propose that anti-correlation in the degree distribution may be a computational strategy employed by sensory cortices to increase the detectability of external stimuli. We show that networks with anti-correlated degrees can in principle be formed by applying learning rules comprised of a combination of spike-timing dependent plasticity, homeostatic plasticity and pruning to networks with uncorrelated degrees. To test our prediction we suggest a novel experimental method to estimate correlations in the degree distribution.

Application of Concepts from Cross-Recurrence Analysis in Speech Production: An Overview and Comparison with Other Nonlinear Methods

ERIC Educational Resources Information Center

Lancia, Leonardo; Fuchs, Susanne; Tiede, Mark

2014-01-01

Purpose: The aim of this article was to introduce an important tool, cross-recurrence analysis, to speech production applications by showing how it can be adapted to evaluate the similarity of multivariate patterns of articulatory motion. The method differs from classical applications of cross-recurrence analysis because no phase space…

The super-Turing computational power of plastic recurrent neural networks.

PubMed

Cabessa, Jérémie; Siegelmann, Hava T

2014-12-01

We study the computational capabilities of a biologically inspired neural model where the synaptic weights, the connectivity pattern, and the number of neurons can evolve over time rather than stay static. Our study focuses on the mere concept of plasticity of the model so that the nature of the updates is assumed to be not constrained. In this context, we show that the so-called plastic recurrent neural networks (RNNs) are capable of the precise super-Turing computational power--as the static analog neural networks--irrespective of whether their synaptic weights are modeled by rational or real numbers, and moreover, irrespective of whether their patterns of plasticity are restricted to bi-valued updates or expressed by any other more general form of updating. Consequently, the incorporation of only bi-valued plastic capabilities in a basic model of RNNs suffices to break the Turing barrier and achieve the super-Turing level of computation. The consideration of more general mechanisms of architectural plasticity or of real synaptic weights does not further increase the capabilities of the networks. These results support the claim that the general mechanism of plasticity is crucially involved in the computational and dynamical capabilities of biological neural networks. They further show that the super-Turing level of computation reflects in a suitable way the capabilities of brain-like models of computation.

The experimental identification of magnetorheological dampers and evaluation of their controllers

NASA Astrophysics Data System (ADS)

Metered, H.; Bonello, P.; Oyadiji, S. O.

2010-05-01

Magnetorheological (MR) fluid dampers are semi-active control devices that have been applied over a wide range of practical vibration control applications. This paper concerns the experimental identification of the dynamic behaviour of an MR damper and the use of the identified parameters in the control of such a damper. Feed-forward and recurrent neural networks are used to model both the direct and inverse dynamics of the damper. Training and validation of the proposed neural networks are achieved by using the data generated through dynamic tests with the damper mounted on a tensile testing machine. The validation test results clearly show that the proposed neural networks can reliably represent both the direct and inverse dynamic behaviours of an MR damper. The effect of the cylinder's surface temperature on both the direct and inverse dynamics of the damper is studied, and the neural network model is shown to be reasonably robust against significant temperature variation. The inverse recurrent neural network model is introduced as a damper controller and experimentally evaluated against alternative controllers proposed in the literature. The results reveal that the neural-based damper controller offers superior damper control. This observation and the added advantages of low-power requirement, extended service life of the damper and the minimal use of sensors, indicate that a neural-based damper controller potentially offers the most cost-effective vibration control solution among the controllers investigated.

The Role of Recurrence Plots in Characterizing the Output-Unemployment Relationship: An Analysis

PubMed Central

Caraiani, Petre; Haven, Emmanuel

2013-01-01

We analyse the output-unemployment relationship using an approach based on cross-recurrence plots and quantitative recurrence analysis. We use post-war period quarterly U.S. data. The results obtained show the emergence of a complex and interesting relationship. PMID:23460814

UArizona at the CLEF eRisk 2017 Pilot Task: Linear and Recurrent Models for Early Depression Detection

PubMed Central

Sadeque, Farig; Xu, Dongfang; Bethard, Steven

2017-01-01

The 2017 CLEF eRisk pilot task focuses on automatically detecting depression as early as possible from a users’ posts to Reddit. In this paper we present the techniques employed for the University of Arizona team’s participation in this early risk detection shared task. We leveraged external information beyond the small training set, including a preexisting depression lexicon and concepts from the Unified Medical Language System as features. For prediction, we used both sequential (recurrent neural network) and non-sequential (support vector machine) models. Our models perform decently on the test data, and the recurrent neural models perform better than the non-sequential support vector machines while using the same feature sets. PMID:29075167

Multiplex visibility graphs to investigate recurrent neural network dynamics

NASA Astrophysics Data System (ADS)

Bianchi, Filippo Maria; Livi, Lorenzo; Alippi, Cesare; Jenssen, Robert

2017-03-01

A recurrent neural network (RNN) is a universal approximator of dynamical systems, whose performance often depends on sensitive hyperparameters. Tuning them properly may be difficult and, typically, based on a trial-and-error approach. In this work, we adopt a graph-based framework to interpret and characterize internal dynamics of a class of RNNs called echo state networks (ESNs). We design principled unsupervised methods to derive hyperparameters configurations yielding maximal ESN performance, expressed in terms of prediction error and memory capacity. In particular, we propose to model time series generated by each neuron activations with a horizontal visibility graph, whose topological properties have been shown to be related to the underlying system dynamics. Successively, horizontal visibility graphs associated with all neurons become layers of a larger structure called a multiplex. We show that topological properties of such a multiplex reflect important features of ESN dynamics that can be used to guide the tuning of its hyperparamers. Results obtained on several benchmarks and a real-world dataset of telephone call data records show the effectiveness of the proposed methods.

Innovative second-generation wavelets construction with recurrent neural networks for solar radiation forecasting.

PubMed

Capizzi, Giacomo; Napoli, Christian; Bonanno, Francesco

2012-11-01

Solar radiation prediction is an important challenge for the electrical engineer because it is used to estimate the power developed by commercial photovoltaic modules. This paper deals with the problem of solar radiation prediction based on observed meteorological data. A 2-day forecast is obtained by using novel wavelet recurrent neural networks (WRNNs). In fact, these WRNNS are used to exploit the correlation between solar radiation and timescale-related variations of wind speed, humidity, and temperature. The input to the selected WRNN is provided by timescale-related bands of wavelet coefficients obtained from meteorological time series. The experimental setup available at the University of Catania, Italy, provided this information. The novelty of this approach is that the proposed WRNN performs the prediction in the wavelet domain and, in addition, also performs the inverse wavelet transform, giving the predicted signal as output. The obtained simulation results show a very low root-mean-square error compared to the results of the solar radiation prediction approaches obtained by hybrid neural networks reported in the recent literature.

Prediction of Sea Surface Temperature Using Long Short-Term Memory

NASA Astrophysics Data System (ADS)

Zhang, Qin; Wang, Hui; Dong, Junyu; Zhong, Guoqiang; Sun, Xin

2017-10-01

This letter adopts long short-term memory(LSTM) to predict sea surface temperature(SST), which is the first attempt, to our knowledge, to use recurrent neural network to solve the problem of SST prediction, and to make one week and one month daily prediction. We formulate the SST prediction problem as a time series regression problem. LSTM is a special kind of recurrent neural network, which introduces gate mechanism into vanilla RNN to prevent the vanished or exploding gradient problem. It has strong ability to model the temporal relationship of time series data and can handle the long-term dependency problem well. The proposed network architecture is composed of two kinds of layers: LSTM layer and full-connected dense layer. LSTM layer is utilized to model the time series relationship. Full-connected layer is utilized to map the output of LSTM layer to a final prediction. We explore the optimal setting of this architecture by experiments and report the accuracy of coastal seas of China to confirm the effectiveness of the proposed method. In addition, we also show its online updated characteristics.

Does money matter in inflation forecasting?

NASA Astrophysics Data System (ADS)

Binner, J. M.; Tino, P.; Tepper, J.; Anderson, R.; Jones, B.; Kendall, G.

2010-11-01

This paper provides the most fully comprehensive evidence to date on whether or not monetary aggregates are valuable for forecasting US inflation in the early to mid 2000s. We explore a wide range of different definitions of money, including different methods of aggregation and different collections of included monetary assets. In our forecasting experiment we use two nonlinear techniques, namely, recurrent neural networks and kernel recursive least squares regression-techniques that are new to macroeconomics. Recurrent neural networks operate with potentially unbounded input memory, while the kernel regression technique is a finite memory predictor. The two methodologies compete to find the best fitting US inflation forecasting models and are then compared to forecasts from a naïve random walk model. The best models were nonlinear autoregressive models based on kernel methods. Our findings do not provide much support for the usefulness of monetary aggregates in forecasting inflation. Beyond its economic findings, our study is in the tradition of physicists’ long-standing interest in the interconnections among statistical mechanics, neural networks, and related nonparametric statistical methods, and suggests potential avenues of extension for such studies.

Multiplex visibility graphs to investigate recurrent neural network dynamics

PubMed Central

Bianchi, Filippo Maria; Livi, Lorenzo; Alippi, Cesare; Jenssen, Robert

2017-01-01

A recurrent neural network (RNN) is a universal approximator of dynamical systems, whose performance often depends on sensitive hyperparameters. Tuning them properly may be difficult and, typically, based on a trial-and-error approach. In this work, we adopt a graph-based framework to interpret and characterize internal dynamics of a class of RNNs called echo state networks (ESNs). We design principled unsupervised methods to derive hyperparameters configurations yielding maximal ESN performance, expressed in terms of prediction error and memory capacity. In particular, we propose to model time series generated by each neuron activations with a horizontal visibility graph, whose topological properties have been shown to be related to the underlying system dynamics. Successively, horizontal visibility graphs associated with all neurons become layers of a larger structure called a multiplex. We show that topological properties of such a multiplex reflect important features of ESN dynamics that can be used to guide the tuning of its hyperparamers. Results obtained on several benchmarks and a real-world dataset of telephone call data records show the effectiveness of the proposed methods. PMID:28281563

Clinical analysis of patients with hepatocellular carcinoma recurrence after living-donor liver transplantation

PubMed Central

Na, Gun Hyung; Hong, Tae Ho; You, Young Kyoung; Kim, Dong Goo

2016-01-01

AIM: To evaluated patterns and outcomes of hepatocellular carcinoma (HCC) recurrence after living donor liver transplantation (LDLT). METHODS: From 2001 to 2014, 293 patients underwent LDLT for HCC at our transplant center. We retrospectively reviewed 54 (18.4%) patients with HCC recurrence after LDLT. We evaluated patterns and outcomes of HCC recurrence after LDLT, with particular attention to the Milan criteria at transplantation, treatments for HCC-recurrent patients, and factors related to survival after HCC recurrence. Furthermore, we evaluated the efficacy of combination treatment of sorafenib and an mTOR inhibitor. RESULTS: The 1-, 2-, and 3-year overall survival rates after HCC recurrence were 41.1%, 20.5%, and 15.4%, respectively. The median time interval between LDLT and HCC recurrence was 6.5 mo. Although recurrence rates according to the Milan criteria at LDLT were significantly different, HCC recurrence patterns and survival rates after HCC recurrence were not significantly different between the two groups. Time to recurrence < 12 mo (P = 0.048), multiple recurrences at HCC recurrence (P = 0.038), and palliative treatment for recurrent tumors (P = 0.003) were significant independent prognostic factors for poor survival after HCC recurrence in a multivariate analysis. The combination treatment of sorafenib and sirolimus showed survival benefits in the palliative treatment group (P = 0.005). CONCLUSION: Curative treatment for recurrent HCC after LDLT is the most important factor in survival rates after HCC recurrence and combination treatments of sorafenib and an mTOR inhibitor could have survival benefits in patients with HCC recurrence after LT in the palliative treatment group. PMID:27433092

Clinical analysis of patients with hepatocellular carcinoma recurrence after living-donor liver transplantation.

PubMed

Na, Gun Hyung; Hong, Tae Ho; You, Young Kyoung; Kim, Dong Goo

2016-07-07

To evaluated patterns and outcomes of hepatocellular carcinoma (HCC) recurrence after living donor liver transplantation (LDLT). From 2001 to 2014, 293 patients underwent LDLT for HCC at our transplant center. We retrospectively reviewed 54 (18.4%) patients with HCC recurrence after LDLT. We evaluated patterns and outcomes of HCC recurrence after LDLT, with particular attention to the Milan criteria at transplantation, treatments for HCC-recurrent patients, and factors related to survival after HCC recurrence. Furthermore, we evaluated the efficacy of combination treatment of sorafenib and an mTOR inhibitor. The 1-, 2-, and 3-year overall survival rates after HCC recurrence were 41.1%, 20.5%, and 15.4%, respectively. The median time interval between LDLT and HCC recurrence was 6.5 mo. Although recurrence rates according to the Milan criteria at LDLT were significantly different, HCC recurrence patterns and survival rates after HCC recurrence were not significantly different between the two groups. Time to recurrence < 12 mo (P = 0.048), multiple recurrences at HCC recurrence (P = 0.038), and palliative treatment for recurrent tumors (P = 0.003) were significant independent prognostic factors for poor survival after HCC recurrence in a multivariate analysis. The combination treatment of sorafenib and sirolimus showed survival benefits in the palliative treatment group (P = 0.005). Curative treatment for recurrent HCC after LDLT is the most important factor in survival rates after HCC recurrence and combination treatments of sorafenib and an mTOR inhibitor could have survival benefits in patients with HCC recurrence after LT in the palliative treatment group.

Factors predicting recurrence of chronic subdural haematoma: the influence of intraoperative irrigation and low-molecular-weight heparin thromboprophylaxis.

PubMed

Tahsim-Oglou, Yasemin; Beseoglu, Kerim; Hänggi, Daniel; Stummer, Walter; Steiger, Hans-Jakob

2012-06-01

Burr-hole drainage has become the accepted treatment of choice for chronic subdural haematoma (cSDH), although still burdened with a major recurrence rate. The current analysis was initiated to determine management-related risk factors for recurrence, i.e. postoperative low-molecular-weight heparin thromboprophylaxis, and the importance of rinsing the subdural space. Two-hundred and forty-seven patients with computerised tomography (CT) defined symptomatic cSDH were managed by two burr-hole trepanations and drainage between January 2005 and November 2008. Postoperative thromboprophylaxis with 40 mg enoxaparine daily was given only during the first half of the study period. For the current analysis the amount of rinsing fluid, postoperative low-dose thromboprophylaxis, as well as age and gender, bilaterality, preoperative and postoperative blood coagulation studies, platelet counts and decrease of subdural fluid on early postoperative CT, were recorded and correlated with recurrence. Statistical calculation was done by univariate and multivariate analysis. A total of 62 of 247 patients needed revision surgery for recurrence (25.1 %). Recurrence rates were significantly lower in the patients treated without postoperative enoxaparine (18.84 %) than in the group with postoperative low-dose enoxaparine thromboprophylaxis (32.11 %) and enoxaparine was administered in a higher proportion of the patients suffering recurrence (P = 0.013). A median intraoperative irrigation volume of 863 ml saline was used in the patients suffering recurrence and 1,500 ml in patients without recurrence (P < 0.001). The median age was slightly higher in the patients suffering from recurrence. Male gender predominated in both groups but was slightly more pronounced in the recurrence group. Preoperative and postoperative platelet counts and plasmatic coagulation indices did not differ significantly between the groups. Relative residual subdural fluid collection on early postoperative CT remained larger in patients finally suffering recurrence (P = 0.03). Multivariate analysis confirmed a small amount of rinsing fluid, male gender and the use of enoxaparine as the most important risk factors for recurrence, although that latter factor did not reach statistical significance in the multivariate analysis. The investigation provides evidence that copious intraoperative irrigation and avoidance of postoperative low-molecular-weight heparin thromboprophylaxis may reduce the recurrence rate of cSDH.

Recurrent Coupling Improves Discrimination of Temporal Spike Patterns

PubMed Central

Yuan, Chun-Wei; Leibold, Christian

2012-01-01

Despite the ubiquitous presence of recurrent synaptic connections in sensory neuronal systems, their general functional purpose is not well understood. A recent conceptual advance has been achieved by theories of reservoir computing in which recurrent networks have been proposed to generate short-term memory as well as to improve neuronal representation of the sensory input for subsequent computations. Here, we present a numerical study on the distinct effects of inhibitory and excitatory recurrence in a canonical linear classification task. It is found that both types of coupling improve the ability to discriminate temporal spike patterns as compared to a purely feed-forward system, although in different ways. For a large class of inhibitory networks, the network’s performance is optimal as long as a fraction of roughly 50% of neurons per stimulus is active in the resulting population code. Thereby the contribution of inactive neurons to the neural code is found to be even more informative than that of the active neurons, generating an inherent robustness of classification performance against temporal jitter of the input spikes. Excitatory couplings are found to not only produce a short-term memory buffer but also to improve linear separability of the population patterns by evoking more irregular firing as compared to the purely inhibitory case. As the excitatory connectivity becomes more sparse, firing becomes more variable, and pattern separability improves. We argue that the proposed paradigm is particularly well-suited as a conceptual framework for processing of sensory information in the auditory pathway. PMID:22586392

Genetics Home Reference: lattice corneal dystrophy type I

MedlinePlus

... have recurrent corneal erosions, which are caused by separation of particular layers of the cornea from one ... intricate network that forms in the spaces between cells and provides structural support to tissues. The protein ...

LSTM-CRF | Informatics Technology for Cancer Research (ITCR)

Cancer.gov

LSTM-CRF uses Natural Language Processing methods for detecting Adverse Drug Events, Drugname, Indication and other medically relevant information from Electronic Health Records. It implements Recurrent Neural Networks using several CRF based inference methods.

Reinforcement Learning of Linking and Tracing Contours in Recurrent Neural Networks

PubMed Central

Brosch, Tobias; Neumann, Heiko; Roelfsema, Pieter R.

2015-01-01

The processing of a visual stimulus can be subdivided into a number of stages. Upon stimulus presentation there is an early phase of feedforward processing where the visual information is propagated from lower to higher visual areas for the extraction of basic and complex stimulus features. This is followed by a later phase where horizontal connections within areas and feedback connections from higher areas back to lower areas come into play. In this later phase, image elements that are behaviorally relevant are grouped by Gestalt grouping rules and are labeled in the cortex with enhanced neuronal activity (object-based attention in psychology). Recent neurophysiological studies revealed that reward-based learning influences these recurrent grouping processes, but it is not well understood how rewards train recurrent circuits for perceptual organization. This paper examines the mechanisms for reward-based learning of new grouping rules. We derive a learning rule that can explain how rewards influence the information flow through feedforward, horizontal and feedback connections. We illustrate the efficiency with two tasks that have been used to study the neuronal correlates of perceptual organization in early visual cortex. The first task is called contour-integration and demands the integration of collinear contour elements into an elongated curve. We show how reward-based learning causes an enhancement of the representation of the to-be-grouped elements at early levels of a recurrent neural network, just as is observed in the visual cortex of monkeys. The second task is curve-tracing where the aim is to determine the endpoint of an elongated curve composed of connected image elements. If trained with the new learning rule, neural networks learn to propagate enhanced activity over the curve, in accordance with neurophysiological data. We close the paper with a number of model predictions that can be tested in future neurophysiological and computational studies. PMID:26496502

A biologically inspired neural network for dynamic programming.

PubMed

Francelin Romero, R A; Kacpryzk, J; Gomide, F

2001-12-01

An artificial neural network with a two-layer feedback topology and generalized recurrent neurons, for solving nonlinear discrete dynamic optimization problems, is developed. A direct method to assign the weights of neural networks is presented. The method is based on Bellmann's Optimality Principle and on the interchange of information which occurs during the synaptic chemical processing among neurons. The neural network based algorithm is an advantageous approach for dynamic programming due to the inherent parallelism of the neural networks; further it reduces the severity of computational problems that can occur in methods like conventional methods. Some illustrative application examples are presented to show how this approach works out including the shortest path and fuzzy decision making problems.

Exponential stabilization and synchronization for fuzzy model of memristive neural networks by periodically intermittent control.

PubMed

Yang, Shiju; Li, Chuandong; Huang, Tingwen

2016-03-01

The problem of exponential stabilization and synchronization for fuzzy model of memristive neural networks (MNNs) is investigated by using periodically intermittent control in this paper. Based on the knowledge of memristor and recurrent neural network, the model of MNNs is formulated. Some novel and useful stabilization criteria and synchronization conditions are then derived by using the Lyapunov functional and differential inequality techniques. It is worth noting that the methods used in this paper are also applied to fuzzy model for complex networks and general neural networks. Numerical simulations are also provided to verify the effectiveness of theoretical results. Copyright © 2015 Elsevier Ltd. All rights reserved.

Y chromosome microdeletions in Mexican males of couples with idiopathic recurrent pregnancy loss.

PubMed

Piña-Aguilar, Raul E; Martínez-Garza, Sandra G; Kohls, Graciela; Vargas-Maciel, Marco A; Vázquez de Lara, Luis G; González-Ortega, Claudia; Cancino-Villarreal, Patricia; Gutiérrez-Gutiérrez, Antonio M

2012-06-01

To analyze the presence of Y chromosome microdeletions in males of Mexican couples with idiopathic recurrent pregnancy losses (RPL). Seventy-one males from couples with RPL and 66 fertile males as controls were studied. DNA was isolated from peripheral lymphocytes and used to run multiplex polymerase chain reactions. Regions AZFa (sY84, sY86), AZFb (sY127, sY134) and AZFc (sY254, sY255) of the Y chromosome were analyzed according to valid guidelines recommended by the European Academy of Andrology and the European Molecular Genetics Quality Network. Also, the sequence tagged sites (STSs): DYS262 (sY67), DYS220 (sY129), DYF85S1 (sY150), DYF86S1 (sY152) and DYF87S1 (sY153) were included in order to analyze STSs previously reported as deleted. A power analysis to support our simple size was performed. Results show an absence of Y chromosome microdeletions in males of couples with RPL and controls with an acceptable statistical power. The study did not show an association of recurrent pregnancy loss and Y chromosome microdeletions in Mexican male partners. Based on the results, the study of Y chromosome microdeletions in couples with RPL is not considered clinically relevant. © 2012 The Authors. Journal of Obstetrics and Gynaecology Research © 2012 Japan Society of Obstetrics and Gynecology.

Use of recurrence plot and recurrence quantification analysis in Taiwan unemployment rate time series

NASA Astrophysics Data System (ADS)

Chen, Wei-Shing

2011-04-01

The aim of the article is to answer the question if the Taiwan unemployment rate dynamics is generated by a non-linear deterministic dynamic process. This paper applies a recurrence plot and recurrence quantification approach based on the analysis of non-stationary hidden transition patterns of the unemployment rate of Taiwan. The case study uses the time series data of the Taiwan’s unemployment rate during the period from 1978/01 to 2010/06. The results show that recurrence techniques are able to identify various phases in the evolution of unemployment transition in Taiwan.

[Clinical target volume delineation for radiotherapy of the esophagus].

PubMed

Lazarescu, I; Thureau, S; Nkhali, L; Pradier, O; Dubray, B

2013-10-01

The dense lymphatic network of the esophagus facilitates tumour spreading along the cephalo-caudal axis and to locoregional lymph nodes. A better understanding of microscopic invasion by tumour cells, based on histological analysis of surgical specimens and analysis of recurrence sites, has justified a reduction in radiotherapy target volumes. The delineation of the clinical target volume (CTV) depends on tumour characteristics (site, histology) and on its spread as assessed on endoscopic ultrasonography and ((18)F)-fluorodeoxyglucose positron-emission tomography (FDG-PET). We propose that positive and negative predictive values for FDG-PET should be used to adapt the CTV according to the risk of nodal involvement. Copyright © 2013 Société française de radiothérapie oncologique (SFRO). Published by Elsevier SAS. All rights reserved.

Deep Recurrent Neural Networks for Human Activity Recognition

PubMed Central

Murad, Abdulmajid

2017-01-01

Adopting deep learning methods for human activity recognition has been effective in extracting discriminative features from raw input sequences acquired from body-worn sensors. Although human movements are encoded in a sequence of successive samples in time, typical machine learning methods perform recognition tasks without exploiting the temporal correlations between input data samples. Convolutional neural networks (CNNs) address this issue by using convolutions across a one-dimensional temporal sequence to capture dependencies among input data. However, the size of convolutional kernels restricts the captured range of dependencies between data samples. As a result, typical models are unadaptable to a wide range of activity-recognition configurations and require fixed-length input windows. In this paper, we propose the use of deep recurrent neural networks (DRNNs) for building recognition models that are capable of capturing long-range dependencies in variable-length input sequences. We present unidirectional, bidirectional, and cascaded architectures based on long short-term memory (LSTM) DRNNs and evaluate their effectiveness on miscellaneous benchmark datasets. Experimental results show that our proposed models outperform methods employing conventional machine learning, such as support vector machine (SVM) and k-nearest neighbors (KNN). Additionally, the proposed models yield better performance than other deep learning techniques, such as deep believe networks (DBNs) and CNNs. PMID:29113103

Deep Recurrent Neural Networks for Human Activity Recognition.

PubMed

Murad, Abdulmajid; Pyun, Jae-Young

2017-11-06

Adopting deep learning methods for human activity recognition has been effective in extracting discriminative features from raw input sequences acquired from body-worn sensors. Although human movements are encoded in a sequence of successive samples in time, typical machine learning methods perform recognition tasks without exploiting the temporal correlations between input data samples. Convolutional neural networks (CNNs) address this issue by using convolutions across a one-dimensional temporal sequence to capture dependencies among input data. However, the size of convolutional kernels restricts the captured range of dependencies between data samples. As a result, typical models are unadaptable to a wide range of activity-recognition configurations and require fixed-length input windows. In this paper, we propose the use of deep recurrent neural networks (DRNNs) for building recognition models that are capable of capturing long-range dependencies in variable-length input sequences. We present unidirectional, bidirectional, and cascaded architectures based on long short-term memory (LSTM) DRNNs and evaluate their effectiveness on miscellaneous benchmark datasets. Experimental results show that our proposed models outperform methods employing conventional machine learning, such as support vector machine (SVM) and k-nearest neighbors (KNN). Additionally, the proposed models yield better performance than other deep learning techniques, such as deep believe networks (DBNs) and CNNs.

EMG-Based Estimation of Limb Movement Using Deep Learning With Recurrent Convolutional Neural Networks.

PubMed

Xia, Peng; Hu, Jie; Peng, Yinghong

2017-10-25

A novel model based on deep learning is proposed to estimate kinematic information for myoelectric control from multi-channel electromyogram (EMG) signals. The neural information of limb movement is embedded in EMG signals that are influenced by all kinds of factors. In order to overcome the negative effects of variability in signals, the proposed model employs the deep architecture combining convolutional neural networks (CNNs) and recurrent neural networks (RNNs). The EMG signals are transformed to time-frequency frames as the input to the model. The limb movement is estimated by the model that is trained with the gradient descent and backpropagation procedure. We tested the model for simultaneous and proportional estimation of limb movement in eight healthy subjects and compared it with support vector regression (SVR) and CNNs on the same data set. The experimental studies show that the proposed model has higher estimation accuracy and better robustness with respect to time. The combination of CNNs and RNNs can improve the model performance compared with using CNNs alone. The model of deep architecture is promising in EMG decoding and optimization of network structures can increase the accuracy and robustness. © 2017 International Center for Artificial Organs and Transplantation and Wiley Periodicals, Inc.

Retrieval of Sentence Sequences for an Image Stream via Coherence Recurrent Convolutional Networks.

PubMed

Park, Cesc Chunseong; Kim, Youngjin; Kim, Gunhee

2018-04-01

We propose an approach for retrieving a sequence of natural sentences for an image stream. Since general users often take a series of pictures on their experiences, much online visual information exists in the form of image streams, for which it would better take into consideration of the whole image stream to produce natural language descriptions. While almost all previous studies have dealt with the relation between a single image and a single natural sentence, our work extends both input and output dimension to a sequence of images and a sequence of sentences. For retrieving a coherent flow of multiple sentences for a photo stream, we propose a multimodal neural architecture called coherence recurrent convolutional network (CRCN), which consists of convolutional neural networks, bidirectional long short-term memory (LSTM) networks, and an entity-based local coherence model. Our approach directly learns from vast user-generated resource of blog posts as text-image parallel training data. We collect more than 22 K unique blog posts with 170 K associated images for the travel topics of NYC, Disneyland , Australia, and Hawaii. We demonstrate that our approach outperforms other state-of-the-art image captioning methods for text sequence generation, using both quantitative measures and user studies via Amazon Mechanical Turk.

A peer review process as part of the implementation of clinical pathways in radiation oncology: Does it improve compliance?

PubMed

Gebhardt, Brian J; Heron, Dwight E; Beriwal, Sushil

Clinical pathways are patient management plans that standardize evidence-based practices to ensure high-quality and cost-effective medical care. Implementation of a pathway is a collaborative process in our network, requiring the active involvement of physicians. This approach promotes acceptance of pathway recommendations, although a peer review process is necessary to ensure compliance and to capture and approve off-pathway selections. We investigated the peer review process and factors associated with time to completion of peer review. Our cancer center implemented radiation oncology pathways for every disease site throughout a large, integrated network. Recommendations are written based upon national guidelines, published literature, and institutional experience with evidence evaluated hierarchically in order of efficacy, toxicity, and then cost. Physicians enter decisions into an online, menu-driven decision support tool that integrates with medical records. Data were collected from the support tool and included the rate of on- and off-pathway selections, peer review decisions performed by disease site directors, and time to complete peer review. A total of 6965 treatment decisions were entered in 2015, and 605 (8.7%) were made off-pathway and were subject to peer review. The median time to peer review decision was 2 days (interquartile range, 0.2-6.8). Factors associated with time to peer review decision >48 hours on univariate analysis include disease site (P < .0001) with a trend toward significance (P = .066) for radiation therapy modality. There was no difference between recurrent and non-recurrent disease (P = .267). Multivariable analysis revealed disease site was associated with time to peer review (P < .001), with lymphoma and skin/sarcoma most strongly influencing decision time >48 hours. Clinical pathways are an integral tool for standardizing evidence-based care throughout our large, integrated network, with 91.3% of all treatment decisions being made as per pathway. The peer review process was feasible, with <1% selections ultimately rejected, suggesting that awareness of peer review of treatment decisions encourages compliance with clinical pathway recommendations. Copyright © 2017 American Society for Radiation Oncology. Published by Elsevier Inc. All rights reserved.

Detecting Lung and Colorectal Cancer Recurrence Using Structured Clinical/Administrative Data to Enable Outcomes Research and Population Health Management.

PubMed

Hassett, Michael J; Uno, Hajime; Cronin, Angel M; Carroll, Nikki M; Hornbrook, Mark C; Ritzwoller, Debra

2017-12-01

Recurrent cancer is common, costly, and lethal, yet we know little about it in community-based populations. Electronic health records and tumor registries contain vast amounts of data regarding community-based patients, but usually lack recurrence status. Existing algorithms that use structured data to detect recurrence have limitations. We developed algorithms to detect the presence and timing of recurrence after definitive therapy for stages I-III lung and colorectal cancer using 2 data sources that contain a widely available type of structured data (claims or electronic health record encounters) linked to gold-standard recurrence status: Medicare claims linked to the Cancer Care Outcomes Research and Surveillance study, and the Cancer Research Network Virtual Data Warehouse linked to registry data. Twelve potential indicators of recurrence were used to develop separate models for each cancer in each data source. Detection models maximized area under the ROC curve (AUC); timing models minimized average absolute error. Algorithms were compared by cancer type/data source, and contrasted with an existing binary detection rule. Detection model AUCs (>0.92) exceeded existing prediction rules. Timing models yielded absolute prediction errors that were small relative to follow-up time (<15%). Similar covariates were included in all detection and timing algorithms, though differences by cancer type and dataset challenged efforts to create 1 common algorithm for all scenarios. Valid and reliable detection of recurrence using big data is feasible. These tools will enable extensive, novel research on quality, effectiveness, and outcomes for lung and colorectal cancer patients and those who develop recurrence.

Local Recurrence After Uveal Melanoma Proton Beam Therapy: Recurrence Types and Prognostic Consequences

DOE Office of Scientific and Technical Information (OSTI.GOV)

Caujolle, Jean-Pierre, E-mail: ncaujolle@aol.com; Paoli, Vincent; Chamorey, Emmanuel

Purpose: To study the prognosis of the different types of uveal melanoma recurrences treated by proton beam therapy (PBT). Methods and Materials: This retrospective study analyzed 61 cases of uveal melanoma local recurrences on a total of 1102 patients treated by PBT between June 1991 and December 2010. Survival rates have been determined by using Kaplan-Meier curves. Prognostic factors have been evaluated by using log-rank test or Cox model. Results: Our local recurrence rate was 6.1% at 5 years. These recurrences were divided into 25 patients with marginal recurrences, 18 global recurrences, 12 distant recurrences, and 6 extrascleral extensions. Fivemore » factors have been identified as statistically significant risk factors of local recurrence in the univariate analysis: large tumoral diameter, small tumoral volume, low ratio of tumoral volume over eyeball volume, iris root involvement, and safety margin inferior to 1 mm. In the local recurrence-free population, the overall survival rate was 68.7% at 10 years and the specific survival rate was 83.6% at 10 years. In the local recurrence population, the overall survival rate was 43.1% at 10 years and the specific survival rate was 55% at 10 years. The multivariate analysis of death risk factors has shown a better prognosis for marginal recurrences. Conclusion: Survival rate of marginal recurrences is superior to that of the other recurrences. The type of recurrence is a clinical prognostic value to take into account. The influence of local recurrence retreatment by proton beam therapy should be evaluated by novel studies.« less

Cascaded bidirectional recurrent neural networks for protein secondary structure prediction.

PubMed

Chen, Jinmiao; Chaudhari, Narendra

2007-01-01

Protein secondary structure (PSS) prediction is an important topic in bioinformatics. Our study on a large set of non-homologous proteins shows that long-range interactions commonly exist and negatively affect PSS prediction. Besides, we also reveal strong correlations between secondary structure (SS) elements. In order to take into account the long-range interactions and SS-SS correlations, we propose a novel prediction system based on cascaded bidirectional recurrent neural network (BRNN). We compare the cascaded BRNN against another two BRNN architectures, namely the original BRNN architecture used for speech recognition as well as Pollastri's BRNN that was proposed for PSS prediction. Our cascaded BRNN achieves an overall three state accuracy Q3 of 74.38\\%, and reaches a high Segment OVerlap (SOV) of 66.0455. It outperforms the original BRNN and Pollastri's BRNN in both Q3 and SOV. Specifically, it improves the SOV score by 4-6%.

Cortical travelling waves: mechanisms and computational principles

PubMed Central

Muller, Lyle; Chavane, Frédéric; Reynolds, John

2018-01-01

Multichannel recording technologies have revealed travelling waves of neural activity in multiple sensory, motor and cognitive systems. These waves can be spontaneously generated by recurrent circuits or evoked by external stimuli. They travel along brain networks at multiple scales, transiently modulating spiking and excitability as they pass. Here, we review recent experimental findings that have found evidence for travelling waves at single-area (mesoscopic) and whole-brain (macroscopic) scales. We place these findings in the context of the current theoretical understanding of wave generation and propagation in recurrent networks. During the large low-frequency rhythms of sleep or the relatively desynchronized state of the awake cortex, travelling waves may serve a variety of functions, from long-term memory consolidation to processing of dynamic visual stimuli. We explore new avenues for experimental and computational understanding of the role of spatiotemporal activity patterns in the cortex. PMID:29563572

Lactate rescues neuronal sodium homeostasis during impaired energy metabolism.

PubMed

Karus, Claudia; Ziemens, Daniel; Rose, Christine R

2015-01-01

Recently, we established that recurrent activity evokes network sodium oscillations in neurons and astrocytes in hippocampal tissue slices. Interestingly, metabolic integrity of astrocytes was essential for the neurons' capacity to maintain low sodium and to recover from sodium loads, indicating an intimate metabolic coupling between the 2 cell types. Here, we studied if lactate can support neuronal sodium homeostasis during impaired energy metabolism by analyzing whether glucose removal, pharmacological inhibition of glycolysis and/or addition of lactate affect cellular sodium regulation. Furthermore, we studied the effect of lactate on sodium regulation during recurrent network activity and upon inhibition of the glial Krebs cycle by sodium-fluoroacetate. Our results indicate that lactate is preferentially used by neurons. They demonstrate that lactate supports neuronal sodium homeostasis and rescues the effects of glial poisoning by sodium-fluoroacetate. Altogether, they are in line with the proposed transfer of lactate from astrocytes to neurons, the so-called astrocyte-neuron-lactate shuttle.

The dynamics of discrete-time computation, with application to recurrent neural networks and finite state machine extraction.

PubMed

Casey, M

1996-08-15

Recurrent neural networks (RNNs) can learn to perform finite state computations. It is shown that an RNN performing a finite state computation must organize its state space to mimic the states in the minimal deterministic finite state machine that can perform that computation, and a precise description of the attractor structure of such systems is given. This knowledge effectively predicts activation space dynamics, which allows one to understand RNN computation dynamics in spite of complexity in activation dynamics. This theory provides a theoretical framework for understanding finite state machine (FSM) extraction techniques and can be used to improve training methods for RNNs performing FSM computations. This provides an example of a successful approach to understanding a general class of complex systems that has not been explicitly designed, e.g., systems that have evolved or learned their internal structure.

Recurrent neural network control for LCC-resonant ultrasonic motor drive.

PubMed

Lin, F J; Wai, R J; Hong, C M

2000-01-01

A newly designed driving circuit for the traveling wave-type ultrasonic motor (USM), which consists of a push-pull DC-DC power converter and a two-phase voltage source inverter using one inductance and two capacitances (LCC) resonant technique, is presented in this study. Moreover, because the dynamic characteristics of the USM are difficult to obtain and the motor parameters are time varying, a recurrent neural network (RNN) controller is proposed to control the USM drive system. In the proposed controller, the dynamic backpropagation algorithm is adopted to train the RNN on-line using the proposed delta adaptation law. Furthermore, to guarantee the convergence of tracking error, analytical methods based on a discrete-type Lyapunov function are proposed to determine the varied learning rates for the training of the RNN. Finally, the effectiveness of the RNN-controlled USM drive system is demonstrated by some experimental results.

Lactate rescues neuronal sodium homeostasis during impaired energy metabolism

PubMed Central

Karus, Claudia; Ziemens, Daniel; Rose, Christine R

2015-01-01

Recently, we established that recurrent activity evokes network sodium oscillations in neurons and astrocytes in hippocampal tissue slices. Interestingly, metabolic integrity of astrocytes was essential for the neurons' capacity to maintain low sodium and to recover from sodium loads, indicating an intimate metabolic coupling between the 2 cell types. Here, we studied if lactate can support neuronal sodium homeostasis during impaired energy metabolism by analyzing whether glucose removal, pharmacological inhibition of glycolysis and/or addition of lactate affect cellular sodium regulation. Furthermore, we studied the effect of lactate on sodium regulation during recurrent network activity and upon inhibition of the glial Krebs cycle by sodium-fluoroacetate. Our results indicate that lactate is preferentially used by neurons. They demonstrate that lactate supports neuronal sodium homeostasis and rescues the effects of glial poisoning by sodium-fluoroacetate. Altogether, they are in line with the proposed transfer of lactate from astrocytes to neurons, the so-called astrocyte-neuron-lactate shuttle. PMID:26039160

Framewise phoneme classification with bidirectional LSTM and other neural network architectures.

PubMed

Graves, Alex; Schmidhuber, Jürgen

2005-01-01

In this paper, we present bidirectional Long Short Term Memory (LSTM) networks, and a modified, full gradient version of the LSTM learning algorithm. We evaluate Bidirectional LSTM (BLSTM) and several other network architectures on the benchmark task of framewise phoneme classification, using the TIMIT database. Our main findings are that bidirectional networks outperform unidirectional ones, and Long Short Term Memory (LSTM) is much faster and also more accurate than both standard Recurrent Neural Nets (RNNs) and time-windowed Multilayer Perceptrons (MLPs). Our results support the view that contextual information is crucial to speech processing, and suggest that BLSTM is an effective architecture with which to exploit it.

Nonlinear adaptive inverse control via the unified model neural network

NASA Astrophysics Data System (ADS)

Jeng, Jin-Tsong; Lee, Tsu-Tian

1999-03-01

In this paper, we propose a new nonlinear adaptive inverse control via a unified model neural network. In order to overcome nonsystematic design and long training time in nonlinear adaptive inverse control, we propose the approximate transformable technique to obtain a Chebyshev Polynomials Based Unified Model (CPBUM) neural network for the feedforward/recurrent neural networks. It turns out that the proposed method can use less training time to get an inverse model. Finally, we apply this proposed method to control magnetic bearing system. The experimental results show that the proposed nonlinear adaptive inverse control architecture provides a greater flexibility and better performance in controlling magnetic bearing systems.

Risk factors of recurrence of macular oedema associated with branch retinal vein occlusion after intravitreal bevacizumab injection.

PubMed

Yoo, Jun Ho; Ahn, Jaemoon; Oh, Jaeryung; Cha, Jaehyung; Kim, Seong-Woo

2017-10-01

To identify risk factors of recurrence of macular oedema in branch retinal vein occlusion (BRVO) after intravitreal bevacizumab (IVB) injection. The records of 63 patients who underwent IVB injection for macular oedema secondary to BRVO with at least 6 months of follow-up were reviewed. Patients were evaluated at baseline with fluorescein angiography (FA), optical coherence tomography (OCT) and ultra-wide-field fundus photography (WFP). During follow-up, OCT and WFP were repeated. The area of retinal haemorrhage, central retinal thickness (CRT), area (mm 2 ) of capillary non-perfusion within the 1 mm (NPA1), 1-3 mm and 6 mm zones of the ETDRS circle, foveal capillary filling time, degree (°) of foveal capillary network destruction and FA pattern were analysed. Macular oedema recurred in 41 of 63 (65.1%) eyes after initial IVB injection. A binary logistic regression model showed that NPA1 (OR=434.97; 95% CI=5.52 to 34262.12, p=0.006) and initial CRT (OR=1.004; 95% CI=1.000 to 1.008, p=0.015) were significantly associated with the recurrence of macular oedema. Receiver operating characteristic curve analysis identified an NPA1 of 0.36 mm 2 (AUC: 0.735, sensitivity: 70.7%; specificity: 63.6%) and an initial CRT of 570 µm (AUC: 0.745, sensitivity: 63.4%; specificity: 77.3%) as cut-off values for predicting recurrence of macular oedema. Patients with BRVO with non-perfusion of more than half of the 1 mm zone of the ETDRS circle or with an initial CRT >570 µm should be closely monitored for macular oedema recurrence within 6 months of IVB injection. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/.

Recurrence interval analysis of trading volumes

NASA Astrophysics Data System (ADS)

Ren, Fei; Zhou, Wei-Xing

2010-06-01

We study the statistical properties of the recurrence intervals τ between successive trading volumes exceeding a certain threshold q . The recurrence interval analysis is carried out for the 20 liquid Chinese stocks covering a period from January 2000 to May 2009, and two Chinese indices from January 2003 to April 2009. Similar to the recurrence interval distribution of the price returns, the tail of the recurrence interval distribution of the trading volumes follows a power-law scaling, and the results are verified by the goodness-of-fit tests using the Kolmogorov-Smirnov (KS) statistic, the weighted KS statistic and the Cramér-von Mises criterion. The measurements of the conditional probability distribution and the detrended fluctuation function show that both short-term and long-term memory effects exist in the recurrence intervals between trading volumes. We further study the relationship between trading volumes and price returns based on the recurrence interval analysis method. It is found that large trading volumes are more likely to occur following large price returns, and the comovement between trading volumes and price returns is more pronounced for large trading volumes.

Recurrence interval analysis of trading volumes.

PubMed

Ren, Fei; Zhou, Wei-Xing

2010-06-01

We study the statistical properties of the recurrence intervals τ between successive trading volumes exceeding a certain threshold q. The recurrence interval analysis is carried out for the 20 liquid Chinese stocks covering a period from January 2000 to May 2009, and two Chinese indices from January 2003 to April 2009. Similar to the recurrence interval distribution of the price returns, the tail of the recurrence interval distribution of the trading volumes follows a power-law scaling, and the results are verified by the goodness-of-fit tests using the Kolmogorov-Smirnov (KS) statistic, the weighted KS statistic and the Cramér-von Mises criterion. The measurements of the conditional probability distribution and the detrended fluctuation function show that both short-term and long-term memory effects exist in the recurrence intervals between trading volumes. We further study the relationship between trading volumes and price returns based on the recurrence interval analysis method. It is found that large trading volumes are more likely to occur following large price returns, and the comovement between trading volumes and price returns is more pronounced for large trading volumes.

A distributed, hierarchical and recurrent framework for reward-based choice

PubMed Central

Hunt, Laurence T.; Hayden, Benjamin Y.

2017-01-01

Many accounts of reward-based choice argue for distinct component processes that are serial and functionally localized. In this article, we argue for an alternative viewpoint, in which choices emerge from repeated computations that are distributed across many brain regions. We emphasize how several features of neuroanatomy may support the implementation of choice, including mutual inhibition in recurrent neural networks and the hierarchical organisation of timescales for information processing across the cortex. This account also suggests that certain correlates of value may be emergent rather than represented explicitly in the brain. PMID:28209978

Towards fully analog hardware reservoir computing for speech recognition

NASA Astrophysics Data System (ADS)

Smerieri, Anteo; Duport, François; Paquot, Yvan; Haelterman, Marc; Schrauwen, Benjamin; Massar, Serge

2012-09-01

Reservoir computing is a very recent, neural network inspired unconventional computation technique, where a recurrent nonlinear system is used in conjunction with a linear readout to perform complex calculations, leveraging its inherent internal dynamics. In this paper we show the operation of an optoelectronic reservoir computer in which both the nonlinear recurrent part and the readout layer are implemented in hardware for a speech recognition application. The performance obtained is close to the one of to state-of-the-art digital reservoirs, while the analog architecture opens the way to ultrafast computation.

Regional Radiation Therapy Impacts Outcome for Node-Positive Cutaneous Melanoma.

PubMed

Strom, Tobin; Torres-Roca, Javier F; Parekh, Akash; Naghavi, Arash O; Caudell, Jimmy J; Oliver, Daniel E; Messina, Jane L; Khushalani, Nikhil I; Zager, Jonathan S; Sarnaik, Amod; Mulé, James J; Trotti, Andy M; Eschrich, Steven A; Sondak, Vernon K; Harrison, Louis B

2017-04-01

Background: Regional radiation therapy (RT) has been shown to reduce the risk of regional recurrence with node-positive cutaneous melanoma. However, risk factors for regional recurrence, especially in the era of sentinel lymph node biopsy (SLNB), are less clear. Our goals were to identify risk factors associated with regional recurrence and to determine whether a radiosensitivity index (RSI) gene expression signature (GES) could identify patients who experience a survival benefit with regional RT. Methods: A single-institution, Institutional Review Board-approved study was performed including 410 patients treated with either SLNB with or without completion lymph node dissection (LND; n=270) or therapeutic LND (n=91). Postoperative regional RT was delivered to the involved nodal basin in 83 cases (20.2%), to a median dose of 54 Gy (range, 30-60 Gy) in 27 fractions (range, 5-30). Primary outcomes were regional control and overall survival by RSI GES status. Results: Median follow-up was 69 months (range, 13-180). Postoperative regional RT was associated with a reduced risk of regional recurrence among all patients on univariate (5-year estimate: 95.0% vs 83.3%; P =.036) and multivariate analysis (hazard ratio[HR], 0.15; 95% CI, 0.05-0.43; P <.001). Among higher-risk subgroups, regional RT was associated with a lower risk of regional recurrence among patients with clinically detected lymph nodes (n=175; 5-year regional control: 94.1% vs 69.5%; P =.003) and extracapsular extension (ECE) present (n=138; 5-year regional control: 96.7% vs 62.2%; P <.001). Among a subset of radiated patients with gene expression data available, a low RSI GES (radiosensitive) tumor status was associated with improved survival compared with a high RSI GES (5-year: 75% vs 0%; HR, 10.68; 95% CI, 1.24-92.14). Conclusions: Regional RT was associated with a reduced risk of regional recurrence among patients with ECE and clinically detected nodal disease. Gene expression data show promise for better predicting radiocurable patients in the future. In the era of increasingly effective systemic therapies, the value of improved regional control potentially takes on greater significance. Copyright © 2017 by the National Comprehensive Cancer Network.

Self-consistent determination of the spike-train power spectrum in a neural network with sparse connectivity.

PubMed

Dummer, Benjamin; Wieland, Stefan; Lindner, Benjamin

2014-01-01

A major source of random variability in cortical networks is the quasi-random arrival of presynaptic action potentials from many other cells. In network studies as well as in the study of the response properties of single cells embedded in a network, synaptic background input is often approximated by Poissonian spike trains. However, the output statistics of the cells is in most cases far from being Poisson. This is inconsistent with the assumption of similar spike-train statistics for pre- and postsynaptic cells in a recurrent network. Here we tackle this problem for the popular class of integrate-and-fire neurons and study a self-consistent statistics of input and output spectra of neural spike trains. Instead of actually using a large network, we use an iterative scheme, in which we simulate a single neuron over several generations. In each of these generations, the neuron is stimulated with surrogate stochastic input that has a similar statistics as the output of the previous generation. For the surrogate input, we employ two distinct approximations: (i) a superposition of renewal spike trains with the same interspike interval density as observed in the previous generation and (ii) a Gaussian current with a power spectrum proportional to that observed in the previous generation. For input parameters that correspond to balanced input in the network, both the renewal and the Gaussian iteration procedure converge quickly and yield comparable results for the self-consistent spike-train power spectrum. We compare our results to large-scale simulations of a random sparsely connected network of leaky integrate-and-fire neurons (Brunel, 2000) and show that in the asynchronous regime close to a state of balanced synaptic input from the network, our iterative schemes provide an excellent approximations to the autocorrelation of spike trains in the recurrent network.

Genome-Based Infection Tracking Reveals Dynamics of Clostridium difficile Transmission and Disease Recurrence.

PubMed

Kumar, Nitin; Miyajima, Fabio; He, Miao; Roberts, Paul; Swale, Andrew; Ellison, Louise; Pickard, Derek; Smith, Godfrey; Molyneux, Rebecca; Dougan, Gordon; Parkhill, Julian; Wren, Brendan W; Parry, Christopher M; Pirmohamed, Munir; Lawley, Trevor D

2016-03-15

Accurate tracking of Clostridium difficile transmission within healthcare settings is key to its containment but is hindered by the lack of discriminatory power of standard genotyping methods. We describe a whole-genome phylogenetic-based method to track the transmission of individual clones in infected hospital patients from the epidemic C. difficile 027/ST1 lineage, and to distinguish between the 2 causes of recurrent disease, relapse (same strain), or reinfection (different strain). We monitored patients with C. difficile infection in a UK hospital over a 2-year period. We performed whole-genome sequencing and phylogenetic analysis of 108 strains isolated from symptomatic patients. High-resolution phylogeny was integrated with in-hospital transfers and contact data to create an infection network linking individual patients and specific hospital wards. Epidemic C. difficile 027/ST1 caused the majority of infections during our sampling period. Integration of whole-genome single nucleotide polymorphism (SNP) phylogenetic analysis, which accurately discriminated between 27 distinct SNP genotypes, with patient movement and contact data identified 32 plausible transmission events, including ward-based contamination (66%) or direct donor-recipient contact (34%). Highly contagious donors were identified who contributed to the persistence of clones within distinct hospital wards and the spread of clones between wards, especially in areas of intense turnover. Recurrent cases were identified between 4 and 26 weeks, highlighting the limitation of the standard <8-week cutoff used for patient diagnosis and management. Genome-based infection tracking to monitor the persistence and spread of C. difficile within healthcare facilities could inform infection control and patient management. © The Author 2015. Published by Oxford University Press for the Infectious Diseases Society of America.

The increased cost of ventral hernia recurrence: a cost analysis.

PubMed

Davila, D G; Parikh, N; Frelich, M J; Goldblatt, M I

2016-12-01

Over 300,000 ventral hernia repairs (VHRs) are performed each year in the US. We sought to assess the economic burden related to ventral hernia recurrences with a focused comparison of those with the initial open versus laparoscopic surgery. The Premier Alliance database from 2009 to 2014 was utilized to obtain patient demographics and comorbid indices, including the Charlson comorbidity index (CCI). Total hospital cost and resource expenses during index laparoscopic and open VHRs and subsequent recurrent repairs were also obtained. The sample was separated into laparoscopic and open repair groups from the initial operation. Adjusted and propensity score matched cost outcome data were then compared amongst groups. One thousand and seventy-seven patients were used for the analysis with a recurrence rate of 3.78 %. For the combined sample, costs were significantly higher during recurrent hernia repair hospitalization ($21,726 versus $19,484, p < 0.0001). However, for index laparoscopic repairs, both the adjusted total hospital cost and department level costs were similar during the index and the recurrent visit. The costs and resource utilization did not go up due to recurrence, even though these patients had greater severity during the recurrent visit (CCI score 0.92 versus 1.06; p = 0.0092). Using a matched sample, the total hospital recurrence cost was higher for the initial open group compared to laparoscopic group ($14,520 versus $12,649; p = 0.0454). Based on our analysis, need for recurrent VHR adds substantially to total hospital costs and resource utilization. Following initial laparoscopic repair, however, the total cost of recurrent repair is not significantly increased, as it is following initial open repair. When comparing the initial laparoscopic repair versus open, the cost of recurrence was higher for the prior open repair group.

Cost-effectiveness of competing strategies for management of recurrent Clostridium difficile infection: a decision analysis.

PubMed

Konijeti, Gauree G; Sauk, Jenny; Shrime, Mark G; Gupta, Meera; Ananthakrishnan, Ashwin N

2014-06-01

Clostridium difficile infection (CDI) is an important cause of morbidity and healthcare costs, and is characterized by high rates of disease recurrence. The cost-effectiveness of newer treatments for recurrent CDI has not been examined, yet would be important to inform clinical practice. The aim of this study was to analyze the cost effectiveness of competing strategies for recurrent CDI. We constructed a decision-analytic model comparing 4 treatment strategies for first-line treatment of recurrent CDI in a population with a median age of 65 years: metronidazole, vancomycin, fidaxomicin, and fecal microbiota transplant (FMT). We modeled up to 2 additional recurrences following the initial recurrence. We assumed FMT delivery via colonoscopy as our base case, but conducted sensitivity analyses based on different modes of delivery. Willingness-to-pay threshold was set at $50 000 per quality-adjusted life-year. At our base case estimates, initial treatment of recurrent CDI using FMT colonoscopy was the most cost-effective strategy, with an incremental cost-effectiveness ratio of $17 016 relative to oral vancomycin. Fidaxomicin and metronidazole were both dominated by FMT colonoscopy. On sensitivity analysis, FMT colonoscopy remained the most cost-effective strategy at cure rates >88.4% and CDI recurrence rates <14.9%. Fidaxomicin required a cost <$1359 to meet our cost-effectiveness threshold. In clinical settings where FMT is not available or applicable, the preferred strategy appears to be initial treatment with oral vancomycin. In this decision analysis examining treatment strategies for recurrent CDI, we demonstrate that FMT colonoscopy is the most cost-effective initial strategy for management of recurrent CDI.

Major prognostic role of Ki67 in localized adrenocortical carcinoma after complete resection.

PubMed

Beuschlein, Felix; Weigel, Jens; Saeger, Wolfgang; Kroiss, Matthias; Wild, Vanessa; Daffara, Fulvia; Libé, Rosella; Ardito, Arianna; Al Ghuzlan, Abir; Quinkler, Marcus; Oßwald, Andrea; Ronchi, Cristina L; de Krijger, Ronald; Feelders, Richard A; Waldmann, Jens; Willenberg, Holger S; Deutschbein, Timo; Stell, Anthony; Reincke, Martin; Papotti, Mauro; Baudin, Eric; Tissier, Frédérique; Haak, Harm R; Loli, Paola; Terzolo, Massimo; Allolio, Bruno; Müller, Hans-Helge; Fassnacht, Martin

2015-03-01

Recurrence of adrenocortical carcinoma (ACC) even after complete (R0) resection occurs frequently. The aim of this study was to identify markers with prognostic value for patients in this clinical setting. From the German ACC registry, 319 patients with the European Network for the Study of Adrenal Tumors stage I-III were identified. As an independent validation cohort, 250 patients from three European countries were included. Clinical, histological, and immunohistochemical markers were correlated with recurrence-free (RFS) and overall survival (OS). Although univariable analysis within the German cohort suggested several factors with potential prognostic power, upon multivariable adjustment only a few including age, tumor size, venous tumor thrombus (VTT), and the proliferation marker Ki67 retained significance. Among these, Ki67 provided the single best prognostic value for RFS (hazard ratio [HR] for recurrence, 1.042 per 1% increase; P < .0001) and OS (HR for death, 1.051; P < .0001) which was confirmed in the validation cohort. Accordingly, clinical outcome differed significantly between patients with Ki67 <10%, 10-19%, and ≥20% (for the German cohort: median RFS, 53.2 vs 31.6 vs 9.4 mo; median OS, 180.5 vs 113.5 vs 42.0 mo). Using the combined cohort prognostic scores including tumor size, VTT, and Ki67 were established. Although these scores discriminated slightly better between subgroups, there was no clinically meaningful advantage in comparison with Ki67 alone. This largest study on prognostic markers in localized ACC identified Ki67 as the single most important factor predicting recurrence in patients following R0 resection. Thus, evaluation of Ki67 indices should be introduced as standard grading in all pathology reports of patients with ACC.

Using deep recurrent neural network for direct beam solar irradiance cloud screening

NASA Astrophysics Data System (ADS)

Chen, Maosi; Davis, John M.; Liu, Chaoshun; Sun, Zhibin; Zempila, Melina Maria; Gao, Wei

2017-09-01

Cloud screening is an essential procedure for in-situ calibration and atmospheric properties retrieval on (UV-)MultiFilter Rotating Shadowband Radiometer [(UV-)MFRSR]. Previous study has explored a cloud screening algorithm for direct-beam (UV-)MFRSR voltage measurements based on the stability assumption on a long time period (typically a half day or a whole day). To design such an algorithm requires in-depth understanding of radiative transfer and delicate data manipulation. Recent rapid developments on deep neural network and computation hardware have opened a window for modeling complicated End-to-End systems with a standardized strategy. In this study, a multi-layer dynamic bidirectional recurrent neural network is built for determining the cloudiness on each time point with a 17-year training dataset and tested with another 1-year dataset. The dataset is the daily 3-minute cosine corrected voltages, airmasses, and the corresponding cloud/clear-sky labels at two stations of the USDA UV-B Monitoring and Research Program. The results show that the optimized neural network model (3-layer, 250 hidden units, and 80 epochs of training) has an overall test accuracy of 97.87% (97.56% for the Oklahoma site and 98.16% for the Hawaii site). Generally, the neural network model grasps the key concept of the original model to use data in the entire day rather than short nearby measurements to perform cloud screening. A scrutiny of the logits layer suggests that the neural network model automatically learns a way to calculate a quantity similar to total optical depth and finds an appropriate threshold for cloud screening.

Genomic analysis reveals secondary glioblastoma after radiotherapy in a subset of recurrent medulloblastomas.

PubMed

Phi, Ji Hoon; Park, Ae Kyung; Lee, Semin; Choi, Seung Ah; Baek, In-Pyo; Kim, Pora; Kim, Eun-Hye; Park, Hee Chul; Kim, Byung Chul; Bhak, Jong; Park, Sung-Hye; Lee, Ji Yeoun; Wang, Kyu-Chang; Kim, Dong-Seok; Shim, Kyu Won; Kim, Se Hoon; Kim, Chae-Yong; Kim, Seung-Ki

2018-06-01

Despite great advances in understanding of molecular pathogenesis and achievement of a high cure rate in medulloblastoma, recurrent medulloblastomas are still dismal. Additionally, misidentification of secondary malignancies due to histological ambiguity leads to misdiagnosis and eventually to inappropriate treatment. Nevertheless, the genomic characteristics of recurrent medulloblastomas are poorly understood, largely due to a lack of matched primary and recurrent tumor tissues. We performed a genomic analysis of recurrent tumors from 17 pediatric medulloblastoma patients. Whole transcriptome sequencing revealed that a subset of recurrent tumors initially diagnosed as locally recurrent medulloblastomas are secondary glioblastomas after radiotherapy, showing high similarity to the non-G-CIMP proneural subtype of glioblastoma. Further analysis, including whole exome sequencing, revealed missense mutations or complex gene fusion events in PDGFRA with augmented expression in the secondary glioblastomas after radiotherapy, implicating PDGFRA as a putative driver in the development of secondary glioblastomas after treatment exposure. This result provides insight into the possible application of PDGFRA-targeted therapy in these second malignancies. Furthermore, genomic alterations of TP53 including 17p loss or germline/somatic mutations were also found in most of the secondary glioblastomas after radiotherapy, indicating a crucial role of TP53 alteration in the process. On the other hand, analysis of recurrent medulloblastomas revealed that the most prevalent alterations are the loss of 17p region including TP53 and gain of 7q region containing EZH2 which already exist in primary tumors. The 7q gain events are frequently accompanied by high expression levels of EZH2 in both primary and recurrent medulloblastomas, which provides a clue to a new therapeutic target to prevent recurrence. Considering the fact that it is often challenging to differentiate between recurrent medulloblastomas and secondary glioblastomas after radiotherapy, our findings have major clinical implications both for correct diagnosis and for potential therapeutic interventions in these devastating diseases.

Multiple μ-stability of neural networks with unbounded time-varying delays.

PubMed

Wang, Lili; Chen, Tianping

2014-05-01

In this paper, we are concerned with a class of recurrent neural networks with unbounded time-varying delays. Based on the geometrical configuration of activation functions, the phase space R(n) can be divided into several Φη-type subsets. Accordingly, a new set of regions Ωη are proposed, and rigorous mathematical analysis is provided to derive the existence of equilibrium point and its local μ-stability in each Ωη. It concludes that the n-dimensional neural networks can exhibit at least 3(n) equilibrium points and 2(n) of them are μ-stable. Furthermore, due to the compatible property, a set of new conditions are presented to address the dynamics in the remaining 3(n)-2(n) subset regions. As direct applications of these results, we can get some criteria on the multiple exponential stability, multiple power stability, multiple log-stability, multiple log-log-stability and so on. In addition, the approach and results can also be extended to the neural networks with K-level nonlinear activation functions and unbounded time-varying delays, in which there can store (2K+1)(n) equilibrium points, (K+1)(n) of them are locally μ-stable. Numerical examples are given to illustrate the effectiveness of our results. Copyright © 2014 Elsevier Ltd. All rights reserved.

A complex network approach for nanoparticle agglomeration analysis in nanoscale images

NASA Astrophysics Data System (ADS)

Machado, Bruno Brandoli; Scabini, Leonardo Felipe; Margarido Orue, Jonatan Patrick; de Arruda, Mauro Santos; Goncalves, Diogo Nunes; Goncalves, Wesley Nunes; Moreira, Raphaell; Rodrigues-Jr, Jose F.

2017-02-01

Complex networks have been widely used in science and technology because of their ability to represent several systems. One of these systems is found in Biochemistry, in which the synthesis of new nanoparticles is a hot topic. However, the interpretation of experimental results in the search of new nanoparticles poses several challenges. This is due to the characteristics of nanoparticle images and due to their multiple intricate properties; one property of recurrent interest is the agglomeration of particles. Addressing this issue, this paper introduces an approach that uses complex networks to detect and describe nanoparticle agglomerates so to foster easier and more insightful analyses. In this approach, each detected particle in an image corresponds to a vertice and the distances between the particles define a criterion for creating edges. Edges are created if the distance is smaller than a radius of interest. Once this network is set, we calculate several discrete measures able to reveal the most outstanding agglomerates in a nanoparticle image. Experimental results using images of scanning tunneling microscopy (STM) of gold nanoparticles demonstrated the effectiveness of the proposed approach over several samples, as reflected by the separability between particles in three usual settings. The results also demonstrated efficacy for both convex and non-convex agglomerates.

The effect of recurrent severe hypoglycemia on cognitive performance in children with type 1 diabetes: a meta-analysis.

PubMed

Blasetti, Annalisa; Chiuri, Rosa Maria; Tocco, Anna Maria; Di Giulio, Concetta; Mattei, Peter A; Ballone, Enzo; Chiarelli, Francesco; Verrotti, Alberto

2011-11-01

The purpose of this study was to investigate the existence and extent of cognitive impairment in type 1 diabetic children with episodes of recurrent severe hypoglycemia, using meta-analysis to synthesize data across studies. The meta-analysis sample included: 441 children with diabetes and recurrent severe hypoglycemia, 560 children with diabetes and without recurrent severe hypoglycemia. Overall, children with type 1 diabetes and recurrent severe hypoglycemia had slightly lower performance than diabetic children without severe hypoglycemia, only in some cognitive domains: intelligence, memory, learning, and verbal fluency/language. Greater impairment was found in memory and learning. No impairment was found for motor speed. Our results seem to confirm the hypothesis that recurrent severe hypoglycemia has a selective negative effect on the children's cognitive functions. However, these results must be considered with caution taking into account factors such as small sample sizes, the different definitions of severe hypoglycemia, and the variety of neuropsychological tests used.

Leveraging blood serotonin as an endophenotype to identify de novo and rare variants involved in autism.

PubMed

Chen, Rui; Davis, Lea K; Guter, Stephen; Wei, Qiang; Jacob, Suma; Potter, Melissa H; Cox, Nancy J; Cook, Edwin H; Sutcliffe, James S; Li, Bingshan

2017-01-01

Autism spectrum disorder (ASD) is one of the most highly heritable neuropsychiatric disorders, but underlying molecular mechanisms are still unresolved due to extreme locus heterogeneity. Leveraging meaningful endophenotypes or biomarkers may be an effective strategy to reduce heterogeneity to identify novel ASD genes. Numerous lines of evidence suggest a link between hyperserotonemia, i.e., elevated serotonin (5-hydroxytryptamine or 5-HT) in whole blood, and ASD. However, the genetic determinants of blood 5-HT level and their relationship to ASD are largely unknown. In this study, pursuing the hypothesis that de novo variants (DNVs) and rare risk alleles acting in a recessive mode may play an important role in predisposition of hyperserotonemia in people with ASD, we carried out whole exome sequencing (WES) in 116 ASD parent-proband trios with most (107) probands having 5-HT measurements. Combined with published ASD DNVs, we identified USP15 as having recurrent de novo loss of function mutations and discovered evidence supporting two other known genes with recurrent DNVs ( FOXP1 and KDM5B ). Genes harboring functional DNVs significantly overlap with functional/disease gene sets known to be involved in ASD etiology, including FMRP targets and synaptic formation and transcriptional regulation genes. We grouped the probands into High-5HT and Normal-5HT groups based on normalized serotonin levels, and used network-based gene set enrichment analysis (NGSEA) to identify novel hyperserotonemia-related ASD genes based on LoF and missense DNVs. We found enrichment in the High-5HT group for a gene network module (DAWN-1) previously implicated in ASD, and this points to the TGF-β pathway and cell junction processes. Through analysis of rare recessively acting variants (RAVs), we also found that rare compound heterozygotes (CHs) in the High-5HT group were enriched for loci in an ASD-associated gene set. Finally, we carried out rare variant group-wise transmission disequilibrium tests (gTDT) and observed significant association of rare variants in genes encoding a subset of the serotonin pathway with ASD. Our study identified USP15 as a novel gene implicated in ASD based on recurrent DNVs. It also demonstrates the potential value of 5-HT as an effective endophenotype for gene discovery in ASD, and the effectiveness of this strategy needs to be further explored in studies of larger sample sizes.

Morphoproteomics, E6/E7 in-situ hybridization, and biomedical analytics define the etiopathogenesis of HPV-associated oropharyngeal carcinoma and provide targeted therapeutic options.

PubMed

Brown, Robert E; Naqvi, Syed; McGuire, Mary F; Buryanek, Jamie; Karni, Ron J

2017-08-17

Human papillomavirus (HPV) has been identified as an etiopathogenetic factor in oropharyngeal squamous cell carcinoma. The HPV E6 and E7 oncogenes are instrumental in promoting proliferation and blocking differentiation leading to tumorigenesis. Although surgical intervention can remove such tumors, the potential for an etiologic field effect with recurrent disease is real. A downstream effector of E7 oncoprotein, enhancer of zeste homolog 2 (EZH2), is known to promote proliferation and to pose a block in differentiation and in turn, could lead to HPV-induced malignant transformation. However, the EZH2 pathway is amenable to low toxicity therapies designed to promote differentiation to a more benign state and prevent recurrent disease by inhibiting the incorporation of HPV into the genome. This is the first study using clinical specimens to demonstrate EZH2 protein expression in oropharyngeal carcinoma (OPC). The study included eight patients with oropharyngeal carcinoma, confirmed p16INK4a- positive by immunohistochemistry (IHC). The tissue expression of E6/E7 messenger RNA (mRNA) was measured by RNAscope® in-situ hybridization technology. Expression of EZH2, Ki-67, and mitotic indices were assessed by morphoproteomic analysis. Biomedical analytics expanded the results with data from Ingenuity Pathway Analysis (IPA) and KEGG databases to construct a molecular network pathway for further insights. Expression of E6 and E7 oncogenes in p16INK4a- positive oropharyngeal carcinoma was confirmed. EZH2 and its correlates, including elevated proliferation index (Ki-67) and mitotic progression were also present. Biomedical analytics validated the relationship between HPV- E6 and E7 and the expression of the EZH2 pathway. There is morphoproteomic and mRNA evidence of the association of p16INK4a-HPV infection with the E6 and E7 oncogenes and the expression of EZH2, Ki-67 and mitotic progression in oropharyngeal carcinoma. The molecular network biology was confirmed by biomedical analytics as consistent with published literature. This is significant because the biology lends itself to targeted therapeutic options using metformin, curcumin, celecoxib and sulforaphane as therapeutic strategies to prevent progression or recurrence of disease.

Ultra-Rapid serial visual presentation reveals dynamics of feedforward and feedback processes in the ventral visual pathway.

PubMed

Mohsenzadeh, Yalda; Qin, Sheng; Cichy, Radoslaw M; Pantazis, Dimitrios

2018-06-21

Human visual recognition activates a dense network of overlapping feedforward and recurrent neuronal processes, making it hard to disentangle processing in the feedforward from the feedback direction. Here, we used ultra-rapid serial visual presentation to suppress sustained activity that blurs the boundaries of processing steps, enabling us to resolve two distinct stages of processing with MEG multivariate pattern classification. The first processing stage was the rapid activation cascade of the bottom-up sweep, which terminated early as visual stimuli were presented at progressively faster rates. The second stage was the emergence of categorical information with peak latency that shifted later in time with progressively faster stimulus presentations, indexing time-consuming recurrent processing. Using MEG-fMRI fusion with representational similarity, we localized recurrent signals in early visual cortex. Together, our findings segregated an initial bottom-up sweep from subsequent feedback processing, and revealed the neural signature of increased recurrent processing demands for challenging viewing conditions. © 2018, Mohsenzadeh et al.

Analog hardware for learning neural networks

NASA Technical Reports Server (NTRS)

Eberhardt, Silvio P. (Inventor)

1991-01-01

This is a recurrent or feedforward analog neural network processor having a multi-level neuron array and a synaptic matrix for storing weighted analog values of synaptic connection strengths which is characterized by temporarily changing one connection strength at a time to determine its effect on system output relative to the desired target. That connection strength is then adjusted based on the effect, whereby the processor is taught the correct response to training examples connection by connection.

"FORCE" learning in recurrent neural networks as data assimilation

NASA Astrophysics Data System (ADS)

Duane, Gregory S.

2017-12-01

It is shown that the "FORCE" algorithm for learning in arbitrarily connected networks of simple neuronal units can be cast as a Kalman Filter, with a particular state-dependent form for the background error covariances. The resulting interpretation has implications for initialization of the learning algorithm, leads to an extension to include interactions between the weight updates for different neurons, and can represent relationships within groups of multiple target output signals.

Are Equilibrium Multichannel Networks Predictable? the Case of the Indus River, Pakistan

NASA Astrophysics Data System (ADS)

Darby, S. E.; Carling, P. A.

2017-12-01

Focusing on the specific case of the Indus River, we argue that the equilibrium planform network structure of large, multi-channel, rivers is predictable. Between Chashma and Taunsa, Pakistan, the Indus is a 264 km long multiple-channel reach. Remote sensing imagery, including a period of time that encompasses the occurrence of major floods in 2007 and 2010, shows that Indus has a minimum of two and a maximum of nine channels, with on average four active channels during the dry season and five during the monsoon. We show that the network structure, if not detailed planform, remains stable, even for the record 2010 flood (27,100 m3s-1; recurrence interval > 100 years). Bankline recession is negligible for discharges less than a peak annual discharge of 6,000 m3s-1 ( 80% of mean annual flow). Maximum Flow Efficiency (MFE) principle demonstrates the channel network is insensitive to the monsoon floods, which typically peak at 13,200 m3s-1. Rather, the network is in near-equilibrium with the mean annual flood (7,530 m3s-1). MFE principle indicates stable networks have three to four channels, thus the observed stability in the number of active channels accords with the presence of a near-equilibrium reach-scale channel network. Insensitivity to the annual hydrological cycle demonstrates that the time-scale for network adjustment is much longer than the time-scale of the monsoon hydrograph, with the annual excess water being stored on floodplains, rather than being conveyed in an enlarged channel network. The analysis explains the lack of significant channel adjustment following the largest flood in 40 years and the extensive Indus flooding experienced on an annual basis, with its substantial impacts on the populace and agricultural production.