The application of neural networks to the SSME startup transient

NASA Technical Reports Server (NTRS)

Meyer, Claudia M.; Maul, William A.

1991-01-01

Feedforward neural networks were used to model three parameters during the Space Shuttle Main Engine startup transient. The three parameters were the main combustion chamber pressure, a controlled parameter, the high pressure oxidizer turbine discharge temperature, a redlined parameter, and the high pressure fuel pump discharge pressure, a failure-indicating performance parameter. Network inputs consisted of time windows of data from engine measurements that correlated highly to the modeled parameter. A standard backpropagation algorithm was used to train the feedforward networks on two nominal firings. Each trained network was validated with four additional nominal firings. For all three parameters, the neural networks were able to accurately predict the data in the validation sets as well as the training set.

Modular, Hierarchical Learning By Artificial Neural Networks

NASA Technical Reports Server (NTRS)

Baldi, Pierre F.; Toomarian, Nikzad

1996-01-01

Modular and hierarchical approach to supervised learning by artificial neural networks leads to neural networks more structured than neural networks in which all neurons fully interconnected. These networks utilize general feedforward flow of information and sparse recurrent connections to achieve dynamical effects. The modular organization, sparsity of modular units and connections, and fact that learning is much more circumscribed are all attractive features for designing neural-network hardware. Learning streamlined by imitating some aspects of biological neural networks.

Forecasting SPEI and SPI Drought Indices Using the Integrated Artificial Neural Networks

PubMed Central

Maca, Petr; Pech, Pavel

2016-01-01

The presented paper compares forecast of drought indices based on two different models of artificial neural networks. The first model is based on feedforward multilayer perceptron, sANN, and the second one is the integrated neural network model, hANN. The analyzed drought indices are the standardized precipitation index (SPI) and the standardized precipitation evaporation index (SPEI) and were derived for the period of 1948–2002 on two US catchments. The meteorological and hydrological data were obtained from MOPEX experiment. The training of both neural network models was made by the adaptive version of differential evolution, JADE. The comparison of models was based on six model performance measures. The results of drought indices forecast, explained by the values of four model performance indices, show that the integrated neural network model was superior to the feedforward multilayer perceptron with one hidden layer of neurons. PMID:26880875

Forecasting SPEI and SPI Drought Indices Using the Integrated Artificial Neural Networks.

PubMed

Maca, Petr; Pech, Pavel

2016-01-01

The presented paper compares forecast of drought indices based on two different models of artificial neural networks. The first model is based on feedforward multilayer perceptron, sANN, and the second one is the integrated neural network model, hANN. The analyzed drought indices are the standardized precipitation index (SPI) and the standardized precipitation evaporation index (SPEI) and were derived for the period of 1948-2002 on two US catchments. The meteorological and hydrological data were obtained from MOPEX experiment. The training of both neural network models was made by the adaptive version of differential evolution, JADE. The comparison of models was based on six model performance measures. The results of drought indices forecast, explained by the values of four model performance indices, show that the integrated neural network model was superior to the feedforward multilayer perceptron with one hidden layer of neurons.

Learning polynomial feedforward neural networks by genetic programming and backpropagation.

PubMed

Nikolaev, N Y; Iba, H

2003-01-01

This paper presents an approach to learning polynomial feedforward neural networks (PFNNs). The approach suggests, first, finding the polynomial network structure by means of a population-based search technique relying on the genetic programming paradigm, and second, further adjustment of the best discovered network weights by an especially derived backpropagation algorithm for higher order networks with polynomial activation functions. These two stages of the PFNN learning process enable us to identify networks with good training as well as generalization performance. Empirical results show that this approach finds PFNN which outperform considerably some previous constructive polynomial network algorithms on processing benchmark time series.

Using an Extended Kalman Filter Learning Algorithm for Feed-Forward Neural Networks to Describe Tracer Correlations

NASA Technical Reports Server (NTRS)

Lary, David J.; Mussa, Yussuf

2004-01-01

In this study a new extended Kalman filter (EKF) learning algorithm for feed-forward neural networks (FFN) is used. With the EKF approach, the training of the FFN can be seen as state estimation for a non-linear stationary process. The EKF method gives excellent convergence performances provided that there is enough computer core memory and that the machine precision is high. Neural networks are ideally suited to describe the spatial and temporal dependence of tracer-tracer correlations. The neural network performs well even in regions where the correlations are less compact and normally a family of correlation curves would be required. For example, the CH4-N2O correlation can be well described using a neural network trained with the latitude, pressure, time of year, and CH4 volume mixing ratio (v.m.r.). The neural network was able to reproduce the CH4-N2O correlation with a correlation coefficient between simulated and training values of 0.9997. The neural network Fortran code used is available for download.

Optimization of Training Sets for Neural-Net Processing of Characteristic Patterns from Vibrating Solids

NASA Technical Reports Server (NTRS)

Decker, Arthur J.

2001-01-01

Artificial neural networks have been used for a number of years to process holography-generated characteristic patterns of vibrating structures. This technology depends critically on the selection and the conditioning of the training sets. A scaling operation called folding is discussed for conditioning training sets optimally for training feed-forward neural networks to process characteristic fringe patterns. Folding allows feed-forward nets to be trained easily to detect damage-induced vibration-displacement-distribution changes as small as 10 nm. A specific application to aerospace of neural-net processing of characteristic patterns is presented to motivate the conditioning and optimization effort.

From biological neural networks to thinking machines: Transitioning biological organizational principles to computer technology

NASA Technical Reports Server (NTRS)

Ross, Muriel D.

1991-01-01

The three-dimensional organization of the vestibular macula is under study by computer assisted reconstruction and simulation methods as a model for more complex neural systems. One goal of this research is to transition knowledge of biological neural network architecture and functioning to computer technology, to contribute to the development of thinking computers. Maculas are organized as weighted neural networks for parallel distributed processing of information. The network is characterized by non-linearity of its terminal/receptive fields. Wiring appears to develop through constrained randomness. A further property is the presence of two main circuits, highly channeled and distributed modifying, that are connected through feedforward-feedback collaterals and biasing subcircuit. Computer simulations demonstrate that differences in geometry of the feedback (afferent) collaterals affects the timing and the magnitude of voltage changes delivered to the spike initiation zone. Feedforward (efferent) collaterals act as voltage followers and likely inhibit neurons of the distributed modifying circuit. These results illustrate the importance of feedforward-feedback loops, of timing, and of inhibition in refining neural network output. They also suggest that it is the distributed modifying network that is most involved in adaptation, memory, and learning. Tests of macular adaptation, through hyper- and microgravitational studies, support this hypothesis since synapses in the distributed modifying circuit, but not the channeled circuit, are altered. Transitioning knowledge of biological systems to computer technology, however, remains problematical.

A Novel Higher Order Artificial Neural Networks

NASA Astrophysics Data System (ADS)

Xu, Shuxiang

2010-05-01

In this paper a new Higher Order Neural Network (HONN) model is introduced and applied in several data mining tasks. Data Mining extracts hidden patterns and valuable information from large databases. A hyperbolic tangent function is used as the neuron activation function for the new HONN model. Experiments are conducted to demonstrate the advantages and disadvantages of the new HONN model, when compared with several conventional Artificial Neural Network (ANN) models: Feedforward ANN with the sigmoid activation function; Feedforward ANN with the hyperbolic tangent activation function; and Radial Basis Function (RBF) ANN with the Gaussian activation function. The experimental results seem to suggest that the new HONN holds higher generalization capability as well as abilities in handling missing data.

Recurrent Convolutional Neural Networks: A Better Model of Biological Object Recognition.

PubMed

Spoerer, Courtney J; McClure, Patrick; Kriegeskorte, Nikolaus

2017-01-01

Feedforward neural networks provide the dominant model of how the brain performs visual object recognition. However, these networks lack the lateral and feedback connections, and the resulting recurrent neuronal dynamics, of the ventral visual pathway in the human and non-human primate brain. Here we investigate recurrent convolutional neural networks with bottom-up (B), lateral (L), and top-down (T) connections. Combining these types of connections yields four architectures (B, BT, BL, and BLT), which we systematically test and compare. We hypothesized that recurrent dynamics might improve recognition performance in the challenging scenario of partial occlusion. We introduce two novel occluded object recognition tasks to test the efficacy of the models, digit clutter (where multiple target digits occlude one another) and digit debris (where target digits are occluded by digit fragments). We find that recurrent neural networks outperform feedforward control models (approximately matched in parametric complexity) at recognizing objects, both in the absence of occlusion and in all occlusion conditions. Recurrent networks were also found to be more robust to the inclusion of additive Gaussian noise. Recurrent neural networks are better in two respects: (1) they are more neurobiologically realistic than their feedforward counterparts; (2) they are better in terms of their ability to recognize objects, especially under challenging conditions. This work shows that computer vision can benefit from using recurrent convolutional architectures and suggests that the ubiquitous recurrent connections in biological brains are essential for task performance.

Criteria for Choosing the Best Neural Network: Part 1

DTIC Science & Technology

1991-07-24

Touretzky, pp. 177-185. San Mateo: Morgan Kaufmann. Harp, S.A., Samad , T., and Guha, A . (1990). Designing application-specific neural networks using genetic...determining a parsimonious neural network for use in prediction/generalization based on a given fixed learning sample. Both the classification and...statistical settings, algorithms for selecting the number of hidden layer nodes in a three layer, feedforward neural network are presented. The selection

Quantum generalisation of feedforward neural networks

NASA Astrophysics Data System (ADS)

Wan, Kwok Ho; Dahlsten, Oscar; Kristjánsson, Hlér; Gardner, Robert; Kim, M. S.

2017-09-01

We propose a quantum generalisation of a classical neural network. The classical neurons are firstly rendered reversible by adding ancillary bits. Then they are generalised to being quantum reversible, i.e., unitary (the classical networks we generalise are called feedforward, and have step-function activation functions). The quantum network can be trained efficiently using gradient descent on a cost function to perform quantum generalisations of classical tasks. We demonstrate numerically that it can: (i) compress quantum states onto a minimal number of qubits, creating a quantum autoencoder, and (ii) discover quantum communication protocols such as teleportation. Our general recipe is theoretical and implementation-independent. The quantum neuron module can naturally be implemented photonically.

Seismic activity prediction using computational intelligence techniques in northern Pakistan

NASA Astrophysics Data System (ADS)

Asim, Khawaja M.; Awais, Muhammad; Martínez-Álvarez, F.; Iqbal, Talat

2017-10-01

Earthquake prediction study is carried out for the region of northern Pakistan. The prediction methodology includes interdisciplinary interaction of seismology and computational intelligence. Eight seismic parameters are computed based upon the past earthquakes. Predictive ability of these eight seismic parameters is evaluated in terms of information gain, which leads to the selection of six parameters to be used in prediction. Multiple computationally intelligent models have been developed for earthquake prediction using selected seismic parameters. These models include feed-forward neural network, recurrent neural network, random forest, multi layer perceptron, radial basis neural network, and support vector machine. The performance of every prediction model is evaluated and McNemar's statistical test is applied to observe the statistical significance of computational methodologies. Feed-forward neural network shows statistically significant predictions along with accuracy of 75% and positive predictive value of 78% in context of northern Pakistan.

Design of Neural Networks for Fast Convergence and Accuracy

NASA Technical Reports Server (NTRS)

Maghami, Peiman G.; Sparks, Dean W., Jr.

1998-01-01

A novel procedure for the design and training of artificial neural networks, used for rapid and efficient controls and dynamics design and analysis for flexible space systems, has been developed. Artificial neural networks are employed to provide a means of evaluating the impact of design changes rapidly. Specifically, two-layer feedforward neural networks are designed to approximate the functional relationship between the component spacecraft design changes and measures of its performance. A training algorithm, based on statistical sampling theory, is presented, which guarantees that the trained networks provide a designer-specified degree of accuracy in mapping the functional relationship. Within each iteration of this statistical-based algorithm, a sequential design algorithm is used for the design and training of the feedforward network to provide rapid convergence to the network goals. Here, at each sequence a new network is trained to minimize the error of previous network. The design algorithm attempts to avoid the local minima phenomenon that hampers the traditional network training. A numerical example is performed on a spacecraft application in order to demonstrate the feasibility of the proposed approach.

Neural networks for feedback feedforward nonlinear control systems.

PubMed

Parisini, T; Zoppoli, R

1994-01-01

This paper deals with the problem of designing feedback feedforward control strategies to drive the state of a dynamic system (in general, nonlinear) so as to track any desired trajectory joining the points of given compact sets, while minimizing a certain cost function (in general, nonquadratic). Due to the generality of the problem, conventional methods are difficult to apply. Thus, an approximate solution is sought by constraining control strategies to take on the structure of multilayer feedforward neural networks. After discussing the approximation properties of neural control strategies, a particular neural architecture is presented, which is based on what has been called the "linear-structure preserving principle". The original functional problem is then reduced to a nonlinear programming one, and backpropagation is applied to derive the optimal values of the synaptic weights. Recursive equations to compute the gradient components are presented, which generalize the classical adjoint system equations of N-stage optimal control theory. Simulation results related to nonlinear nonquadratic problems show the effectiveness of the proposed method.

Deterministic convergence of chaos injection-based gradient method for training feedforward neural networks.

PubMed

Zhang, Huisheng; Zhang, Ying; Xu, Dongpo; Liu, Xiaodong

2015-06-01

It has been shown that, by adding a chaotic sequence to the weight update during the training of neural networks, the chaos injection-based gradient method (CIBGM) is superior to the standard backpropagation algorithm. This paper presents the theoretical convergence analysis of CIBGM for training feedforward neural networks. We consider both the case of batch learning as well as the case of online learning. Under mild conditions, we prove the weak convergence, i.e., the training error tends to a constant and the gradient of the error function tends to zero. Moreover, the strong convergence of CIBGM is also obtained with the help of an extra condition. The theoretical results are substantiated by a simulation example.

Subtractive, divisive and non-monotonic gain control in feedforward nets linearized by noise and delays.

PubMed

Mejias, Jorge F; Payeur, Alexandre; Selin, Erik; Maler, Leonard; Longtin, André

2014-01-01

The control of input-to-output mappings, or gain control, is one of the main strategies used by neural networks for the processing and gating of information. Using a spiking neural network model, we studied the gain control induced by a form of inhibitory feedforward circuitry-also known as "open-loop feedback"-, which has been experimentally observed in a cerebellum-like structure in weakly electric fish. We found, both analytically and numerically, that this network displays three different regimes of gain control: subtractive, divisive, and non-monotonic. Subtractive gain control was obtained when noise is very low in the network. Also, it was possible to change from divisive to non-monotonic gain control by simply modulating the strength of the feedforward inhibition, which may be achieved via long-term synaptic plasticity. The particular case of divisive gain control has been previously observed in vivo in weakly electric fish. These gain control regimes were robust to the presence of temporal delays in the inhibitory feedforward pathway, which were found to linearize the input-to-output mappings (or f-I curves) via a novel variability-increasing mechanism. Our findings highlight the feedforward-induced gain control analyzed here as a highly versatile mechanism of information gating in the brain.

Subtractive, divisive and non-monotonic gain control in feedforward nets linearized by noise and delays

PubMed Central

Mejias, Jorge F.; Payeur, Alexandre; Selin, Erik; Maler, Leonard; Longtin, André

2014-01-01

The control of input-to-output mappings, or gain control, is one of the main strategies used by neural networks for the processing and gating of information. Using a spiking neural network model, we studied the gain control induced by a form of inhibitory feedforward circuitry—also known as “open-loop feedback”—, which has been experimentally observed in a cerebellum-like structure in weakly electric fish. We found, both analytically and numerically, that this network displays three different regimes of gain control: subtractive, divisive, and non-monotonic. Subtractive gain control was obtained when noise is very low in the network. Also, it was possible to change from divisive to non-monotonic gain control by simply modulating the strength of the feedforward inhibition, which may be achieved via long-term synaptic plasticity. The particular case of divisive gain control has been previously observed in vivo in weakly electric fish. These gain control regimes were robust to the presence of temporal delays in the inhibitory feedforward pathway, which were found to linearize the input-to-output mappings (or f-I curves) via a novel variability-increasing mechanism. Our findings highlight the feedforward-induced gain control analyzed here as a highly versatile mechanism of information gating in the brain. PMID:24616694

Development of programmable artificial neural networks

NASA Technical Reports Server (NTRS)

Meade, Andrew J.

1993-01-01

Conventionally programmed digital computers can process numbers with great speed and precision, but do not easily recognize patterns or imprecise or contradictory data. Instead of being programmed in the conventional sense, artificial neural networks are capable of self-learning through exposure to repeated examples. However, the training of an ANN can be a time consuming and unpredictable process. A general method is being developed to mate the adaptability of the ANN with the speed and precision of the digital computer. This method was successful in building feedforward networks that can approximate functions and their partial derivatives from examples in a single iteration. The general method also allows the formation of feedforward networks that can approximate the solution to nonlinear ordinary and partial differential equations to desired accuracy without the need of examples. It is believed that continued research will produce artificial neural networks that can be used with confidence in practical scientific computing and engineering applications.

A feedforward artificial neural network based on quantum effect vector-matrix multipliers.

PubMed

Levy, H J; McGill, T C

1993-01-01

The vector-matrix multiplier is the engine of many artificial neural network implementations because it can simulate the way in which neurons collect weighted input signals from a dendritic arbor. A new technology for building analog weighting elements that is theoretically capable of densities and speeds far beyond anything that conventional VLSI in silicon could ever offer is presented. To illustrate the feasibility of such a technology, a small three-layer feedforward prototype network with five binary neurons and six tri-state synapses was built and used to perform all of the fundamental logic functions: XOR, AND, OR, and NOT.

Feature to prototype transition in neural networks

NASA Astrophysics Data System (ADS)

Krotov, Dmitry; Hopfield, John

Models of associative memory with higher order (higher than quadratic) interactions, and their relationship to neural networks used in deep learning are discussed. Associative memory is conventionally described by recurrent neural networks with dynamical convergence to stable points. Deep learning typically uses feedforward neural nets without dynamics. However, a simple duality relates these two different views when applied to problems of pattern classification. From the perspective of associative memory such models deserve attention because they make it possible to store a much larger number of memories, compared to the quadratic case. In the dual description, these models correspond to feedforward neural networks with one hidden layer and unusual activation functions transmitting the activities of the visible neurons to the hidden layer. These activation functions are rectified polynomials of a higher degree rather than the rectified linear functions used in deep learning. The network learns representations of the data in terms of features for rectified linear functions, but as the power in the activation function is increased there is a gradual shift to a prototype-based representation, the two extreme regimes of pattern recognition known in cognitive psychology. Simons Center for Systems Biology.

Neural Network Classifier Architectures for Phoneme Recognition. CRC Technical Note No. CRC-TN-92-001.

ERIC Educational Resources Information Center

Treurniet, William

A study applied artificial neural networks, trained with the back-propagation learning algorithm, to modelling phonemes extracted from the DARPA TIMIT multi-speaker, continuous speech data base. A number of proposed network architectures were applied to the phoneme classification task, ranging from the simple feedforward multilayer network to more…

Self-growing neural network architecture using crisp and fuzzy entropy

NASA Technical Reports Server (NTRS)

Cios, Krzysztof J.

1992-01-01

The paper briefly describes the self-growing neural network algorithm, CID2, which makes decision trees equivalent to hidden layers of a neural network. The algorithm generates a feedforward architecture using crisp and fuzzy entropy measures. The results of a real-life recognition problem of distinguishing defects in a glass ribbon and of a benchmark problem of differentiating two spirals are shown and discussed.

Self-growing neural network architecture using crisp and fuzzy entropy

NASA Technical Reports Server (NTRS)

Cios, Krzysztof J.

1992-01-01

The paper briefly describes the self-growing neural network algorithm, CID3, which makes decision trees equivalent to hidden layers of a neural network. The algorithm generates a feedforward architecture using crisp and fuzzy entropy measures. The results for a real-life recognition problem of distinguishing defects in a glass ribbon, and for a benchmark problen of telling two spirals apart are shown and discussed.

Neural net diagnostics for VLSI test

NASA Technical Reports Server (NTRS)

Lin, T.; Tseng, H.; Wu, A.; Dogan, N.; Meador, J.

1990-01-01

This paper discusses the application of neural network pattern analysis algorithms to the IC fault diagnosis problem. A fault diagnostic is a decision rule combining what is known about an ideal circuit test response with information about how it is distorted by fabrication variations and measurement noise. The rule is used to detect fault existence in fabricated circuits using real test equipment. Traditional statistical techniques may be used to achieve this goal, but they can employ unrealistic a priori assumptions about measurement data. Our approach to this problem employs an adaptive pattern analysis technique based on feedforward neural networks. During training, a feedforward network automatically captures unknown sample distributions. This is important because distributions arising from the nonlinear effects of process variation can be more complex than is typically assumed. A feedforward network is also able to extract measurement features which contribute significantly to making a correct decision. Traditional feature extraction techniques employ matrix manipulations which can be particularly costly for large measurement vectors. In this paper we discuss a software system which we are developing that uses this approach. We also provide a simple example illustrating the use of the technique for fault detection in an operational amplifier.

Temporal neural networks and transient analysis of complex engineering systems

NASA Astrophysics Data System (ADS)

Uluyol, Onder

A theory is introduced for a multi-layered Local Output Gamma Feedback (LOGF) neural network within the paradigm of Locally-Recurrent Globally-Feedforward neural networks. It is developed for the identification, prediction, and control tasks of spatio-temporal systems and allows for the presentation of different time scales through incorporation of a gamma memory. It is initially applied to the tasks of sunspot and Mackey-Glass series prediction as benchmarks, then it is extended to the task of power level control of a nuclear reactor at different fuel cycle conditions. The developed LOGF neuron model can also be viewed as a Transformed Input and State (TIS) Gamma memory for neural network architectures for temporal processing. The novel LOGF neuron model extends the static neuron model by incorporating into it a short-term memory structure in the form of a digital gamma filter. A feedforward neural network made up of LOGF neurons can thus be used to model dynamic systems. A learning algorithm based upon the Backpropagation-Through-Time (BTT) approach is derived. It is applicable for training a general L-layer LOGF neural network. The spatial and temporal weights and parameters of the network are iteratively optimized for a given problem using the derived learning algorithm.

Noise Tolerance of Attractor and Feedforward Memory Models

PubMed Central

Lim, Sukbin; Goldman, Mark S.

2017-01-01

In short-term memory networks, transient stimuli are represented by patterns of neural activity that persist long after stimulus offset. Here, we compare the performance of two prominent classes of memory networks, feedback-based attractor networks and feedforward networks, in conveying information about the amplitude of a briefly presented stimulus in the presence of gaussian noise. Using Fisher information as a metric of memory performance, we find that the optimal form of network architecture depends strongly on assumptions about the forms of nonlinearities in the network. For purely linear networks, we find that feedforward networks outperform attractor networks because noise is continually removed from feedforward networks when signals exit the network; as a result, feedforward networks can amplify signals they receive faster than noise accumulates over time. By contrast, attractor networks must operate in a signal-attenuating regime to avoid the buildup of noise. However, if the amplification of signals is limited by a finite dynamic range of neuronal responses or if noise is reset at the time of signal arrival, as suggested by recent experiments, we find that attractor networks can out-perform feedforward ones. Under a simple model in which neurons have a finite dynamic range, we find that the optimal attractor networks are forgetful if there is no mechanism for noise reduction with signal arrival but nonforgetful (perfect integrators) in the presence of a strong reset mechanism. Furthermore, we find that the maximal Fisher information for the feedforward and attractor networks exhibits power law decay as a function of time and scales linearly with the number of neurons. These results highlight prominent factors that lead to trade-offs in the memory performance of networks with different architectures and constraints, and suggest conditions under which attractor or feedforward networks may be best suited to storing information about previous stimuli. PMID:22091664

Feed-forward neural network model for hunger and satiety related VAS score prediction.

PubMed

Krishnan, Shaji; Hendriks, Henk F J; Hartvigsen, Merete L; de Graaf, Albert A

2016-07-07

An artificial neural network approach was chosen to model the outcome of the complex signaling pathways in the gastro-intestinal tract and other peripheral organs that eventually produce the satiety feeling in the brain upon feeding. A multilayer feed-forward neural network was trained with sets of experimental data relating concentration-time courses of plasma satiety hormones to Visual Analog Scales (VAS) scores. The network successfully predicted VAS responses from sets of satiety hormone data obtained in experiments using different food compositions. The correlation coefficients for the predicted VAS responses for test sets having i) a full set of three satiety hormones, ii) a set of only two satiety hormones, and iii) a set of only one satiety hormone were 0.96, 0.96, and 0.89, respectively. The predicted VAS responses discriminated the satiety effects of high satiating food types from less satiating food types both in orally fed and ileal infused forms. From this application of artificial neural networks, one may conclude that neural network models are very suitable to describe situations where behavior is complex and incompletely understood. However, training data sets that fit the experimental conditions need to be available.

Numerical solution of the nonlinear Schrodinger equation by feedforward neural networks

NASA Astrophysics Data System (ADS)

Shirvany, Yazdan; Hayati, Mohsen; Moradian, Rostam

2008-12-01

We present a method to solve boundary value problems using artificial neural networks (ANN). A trial solution of the differential equation is written as a feed-forward neural network containing adjustable parameters (the weights and biases). From the differential equation and its boundary conditions we prepare the energy function which is used in the back-propagation method with momentum term to update the network parameters. We improved energy function of ANN which is derived from Schrodinger equation and the boundary conditions. With this improvement of energy function we can use unsupervised training method in the ANN for solving the equation. Unsupervised training aims to minimize a non-negative energy function. We used the ANN method to solve Schrodinger equation for few quantum systems. Eigenfunctions and energy eigenvalues are calculated. Our numerical results are in agreement with their corresponding analytical solution and show the efficiency of ANN method for solving eigenvalue problems.

Deep Neural Networks: A New Framework for Modeling Biological Vision and Brain Information Processing.

PubMed

Kriegeskorte, Nikolaus

2015-11-24

Recent advances in neural network modeling have enabled major strides in computer vision and other artificial intelligence applications. Human-level visual recognition abilities are coming within reach of artificial systems. Artificial neural networks are inspired by the brain, and their computations could be implemented in biological neurons. Convolutional feedforward networks, which now dominate computer vision, take further inspiration from the architecture of the primate visual hierarchy. However, the current models are designed with engineering goals, not to model brain computations. Nevertheless, initial studies comparing internal representations between these models and primate brains find surprisingly similar representational spaces. With human-level performance no longer out of reach, we are entering an exciting new era, in which we will be able to build biologically faithful feedforward and recurrent computational models of how biological brains perform high-level feats of intelligence, including vision.

Design of Neural Networks for Fast Convergence and Accuracy: Dynamics and Control

NASA Technical Reports Server (NTRS)

Maghami, Peiman G.; Sparks, Dean W., Jr.

1997-01-01

A procedure for the design and training of artificial neural networks, used for rapid and efficient controls and dynamics design and analysis for flexible space systems, has been developed. Artificial neural networks are employed, such that once properly trained, they provide a means of evaluating the impact of design changes rapidly. Specifically, two-layer feedforward neural networks are designed to approximate the functional relationship between the component/spacecraft design changes and measures of its performance or nonlinear dynamics of the system/components. A training algorithm, based on statistical sampling theory, is presented, which guarantees that the trained networks provide a designer-specified degree of accuracy in mapping the functional relationship. Within each iteration of this statistical-based algorithm, a sequential design algorithm is used for the design and training of the feedforward network to provide rapid convergence to the network goals. Here, at each sequence a new network is trained to minimize the error of previous network. The proposed method should work for applications wherein an arbitrary large source of training data can be generated. Two numerical examples are performed on a spacecraft application in order to demonstrate the feasibility of the proposed approach.

Design of neural networks for fast convergence and accuracy: dynamics and control.

PubMed

Maghami, P G; Sparks, D R

2000-01-01

A procedure for the design and training of artificial neural networks, used for rapid and efficient controls and dynamics design and analysis for flexible space systems, has been developed. Artificial neural networks are employed, such that once properly trained, they provide a means of evaluating the impact of design changes rapidly. Specifically, two-layer feedforward neural networks are designed to approximate the functional relationship between the component/spacecraft design changes and measures of its performance or nonlinear dynamics of the system/components. A training algorithm, based on statistical sampling theory, is presented, which guarantees that the trained networks provide a designer-specified degree of accuracy in mapping the functional relationship. Within each iteration of this statistical-based algorithm, a sequential design algorithm is used for the design and training of the feedforward network to provide rapid convergence to the network goals. Here, at each sequence a new network is trained to minimize the error of previous network. The proposed method should work for applications wherein an arbitrary large source of training data can be generated. Two numerical examples are performed on a spacecraft application in order to demonstrate the feasibility of the proposed approach.

A feed-forward Hopfield neural network algorithm (FHNNA) with a colour satellite image for water quality mapping

NASA Astrophysics Data System (ADS)

Asal Kzar, Ahmed; Mat Jafri, M. Z.; Hwee San, Lim; Al-Zuky, Ali A.; Mutter, Kussay N.; Hassan Al-Saleh, Anwar

2016-06-01

There are many techniques that have been given for water quality problem, but the remote sensing techniques have proven their success, especially when the artificial neural networks are used as mathematical models with these techniques. Hopfield neural network is one type of artificial neural networks which is common, fast, simple, and efficient, but it when it deals with images that have more than two colours such as remote sensing images. This work has attempted to solve this problem via modifying the network that deals with colour remote sensing images for water quality mapping. A Feed-forward Hopfield Neural Network Algorithm (FHNNA) was modified and used with a satellite colour image from type of Thailand earth observation system (THEOS) for TSS mapping in the Penang strait, Malaysia, through the classification of TSS concentrations. The new algorithm is based essentially on three modifications: using HNN as feed-forward network, considering the weights of bitplanes, and non-self-architecture or zero diagonal of weight matrix, in addition, it depends on a validation data. The achieved map was colour-coded for visual interpretation. The efficiency of the new algorithm has found out by the higher correlation coefficient (R=0.979) and the lower root mean square error (RMSE=4.301) between the validation data that were divided into two groups. One used for the algorithm and the other used for validating the results. The comparison was with the minimum distance classifier. Therefore, TSS mapping of polluted water in Penang strait, Malaysia, can be performed using FHNNA with remote sensing technique (THEOS). It is a new and useful application of HNN, so it is a new model with remote sensing techniques for water quality mapping which is considered important environmental problem.

Neural networks for function approximation in nonlinear control

NASA Technical Reports Server (NTRS)

Linse, Dennis J.; Stengel, Robert F.

1990-01-01

Two neural network architectures are compared with a classical spline interpolation technique for the approximation of functions useful in a nonlinear control system. A standard back-propagation feedforward neural network and a cerebellar model articulation controller (CMAC) neural network are presented, and their results are compared with a B-spline interpolation procedure that is updated using recursive least-squares parameter identification. Each method is able to accurately represent a one-dimensional test function. Tradeoffs between size requirements, speed of operation, and speed of learning indicate that neural networks may be practical for identification and adaptation in a nonlinear control environment.

A case for spiking neural network simulation based on configurable multiple-FPGA systems.

PubMed

Yang, Shufan; Wu, Qiang; Li, Renfa

2011-09-01

Recent neuropsychological research has begun to reveal that neurons encode information in the timing of spikes. Spiking neural network simulations are a flexible and powerful method for investigating the behaviour of neuronal systems. Simulation of the spiking neural networks in software is unable to rapidly generate output spikes in large-scale of neural network. An alternative approach, hardware implementation of such system, provides the possibility to generate independent spikes precisely and simultaneously output spike waves in real time, under the premise that spiking neural network can take full advantage of hardware inherent parallelism. We introduce a configurable FPGA-oriented hardware platform for spiking neural network simulation in this work. We aim to use this platform to combine the speed of dedicated hardware with the programmability of software so that it might allow neuroscientists to put together sophisticated computation experiments of their own model. A feed-forward hierarchy network is developed as a case study to describe the operation of biological neural systems (such as orientation selectivity of visual cortex) and computational models of such systems. This model demonstrates how a feed-forward neural network constructs the circuitry required for orientation selectivity and provides platform for reaching a deeper understanding of the primate visual system. In the future, larger scale models based on this framework can be used to replicate the actual architecture in visual cortex, leading to more detailed predictions and insights into visual perception phenomenon.

Using Upper Layer Weights to Efficiently Construct and Train Feedforward Neural Networks Executing Backpropagation

DTIC Science & Technology

2011-03-01

algorithm is utilized by Belue, Steppe, & Bauer and Kocur , et al. (Belue, Steppe, & Bauer, April 1996) ( Kocur , et al., 1996). Bacauskiene and...Society. Cardiff, UK. Kocur , C., Roger, S., Myers, L., Burns, T., Hoffmeister, J., Bauer, K., et al. (1996). Using neural networks to select

An Indoor Positioning Technique Based on a Feed-Forward Artificial Neural Network Using Levenberg-Marquardt Learning Method

NASA Astrophysics Data System (ADS)

Pahlavani, P.; Gholami, A.; Azimi, S.

2017-09-01

This paper presents an indoor positioning technique based on a multi-layer feed-forward (MLFF) artificial neural networks (ANN). Most of the indoor received signal strength (RSS)-based WLAN positioning systems use the fingerprinting technique that can be divided into two phases: the offline (calibration) phase and the online (estimation) phase. In this paper, RSSs were collected for all references points in four directions and two periods of time (Morning and Evening). Hence, RSS readings were sampled at a regular time interval and specific orientation at each reference point. The proposed ANN based model used Levenberg-Marquardt algorithm for learning and fitting the network to the training data. This RSS readings in all references points and the known position of these references points was prepared for training phase of the proposed MLFF neural network. Eventually, the average positioning error for this network using 30% check and validation data was computed approximately 2.20 meter.

Numerical solution of differential equations by artificial neural networks

NASA Technical Reports Server (NTRS)

Meade, Andrew J., Jr.

1995-01-01

Conventionally programmed digital computers can process numbers with great speed and precision, but do not easily recognize patterns or imprecise or contradictory data. Instead of being programmed in the conventional sense, artificial neural networks (ANN's) are capable of self-learning through exposure to repeated examples. However, the training of an ANN can be a time consuming and unpredictable process. A general method is being developed by the author to mate the adaptability of the ANN with the speed and precision of the digital computer. This method has been successful in building feedforward networks that can approximate functions and their partial derivatives from examples in a single iteration. The general method also allows the formation of feedforward networks that can approximate the solution to nonlinear ordinary and partial differential equations to desired accuracy without the need of examples. It is believed that continued research will produce artificial neural networks that can be used with confidence in practical scientific computing and engineering applications.

Forecasting the mortality rates of Indonesian population by using neural network

NASA Astrophysics Data System (ADS)

Safitri, Lutfiani; Mardiyati, Sri; Rahim, Hendrisman

2018-03-01

A model that can represent a problem is required in conducting a forecasting. One of the models that has been acknowledged by the actuary community in forecasting mortality rate is the Lee-Certer model. Lee Carter model supported by Neural Network will be used to calculate mortality forecasting in Indonesia. The type of Neural Network used is feedforward neural network aligned with backpropagation algorithm in python programming language. And the final result of this study is mortality rate in forecasting Indonesia for the next few years

Artificial neural networks in Space Station optimal attitude control

NASA Astrophysics Data System (ADS)

Kumar, Renjith R.; Seywald, Hans; Deshpande, Samir M.; Rahman, Zia

1992-08-01

Innovative techniques of using 'Artificial Neural Networks' (ANN) for improving the performance of the pitch axis attitude control system of Space Station Freedom using Control Moment Gyros (CMGs) are investigated. The first technique uses a feedforward ANN with multilayer perceptrons to obtain an on-line controller which improves the performance of the control system via a model following approach. The second techique uses a single layer feedforward ANN with a modified back propagation scheme to estimate the internal plant variations and the external disturbances separately. These estimates are then used to solve two differential Riccati equations to obtain time varying gains which improve the control system performance in successive orbits.

A Spiking Neural Network Based Cortex-Like Mechanism and Application to Facial Expression Recognition

PubMed Central

Fu, Si-Yao; Yang, Guo-Sheng; Kuai, Xin-Kai

2012-01-01

In this paper, we present a quantitative, highly structured cortex-simulated model, which can be simply described as feedforward, hierarchical simulation of ventral stream of visual cortex using biologically plausible, computationally convenient spiking neural network system. The motivation comes directly from recent pioneering works on detailed functional decomposition analysis of the feedforward pathway of the ventral stream of visual cortex and developments on artificial spiking neural networks (SNNs). By combining the logical structure of the cortical hierarchy and computing power of the spiking neuron model, a practical framework has been presented. As a proof of principle, we demonstrate our system on several facial expression recognition tasks. The proposed cortical-like feedforward hierarchy framework has the merit of capability of dealing with complicated pattern recognition problems, suggesting that, by combining the cognitive models with modern neurocomputational approaches, the neurosystematic approach to the study of cortex-like mechanism has the potential to extend our knowledge of brain mechanisms underlying the cognitive analysis and to advance theoretical models of how we recognize face or, more specifically, perceive other people's facial expression in a rich, dynamic, and complex environment, providing a new starting point for improved models of visual cortex-like mechanism. PMID:23193391

Interest rate next-day variation prediction based on hybrid feedforward neural network, particle swarm optimization, and multiresolution techniques

NASA Astrophysics Data System (ADS)

Lahmiri, Salim

2016-02-01

Multiresolution analysis techniques including continuous wavelet transform, empirical mode decomposition, and variational mode decomposition are tested in the context of interest rate next-day variation prediction. In particular, multiresolution analysis techniques are used to decompose interest rate actual variation and feedforward neural network for training and prediction. Particle swarm optimization technique is adopted to optimize its initial weights. For comparison purpose, autoregressive moving average model, random walk process and the naive model are used as main reference models. In order to show the feasibility of the presented hybrid models that combine multiresolution analysis techniques and feedforward neural network optimized by particle swarm optimization, we used a set of six illustrative interest rates; including Moody's seasoned Aaa corporate bond yield, Moody's seasoned Baa corporate bond yield, 3-Month, 6-Month and 1-Year treasury bills, and effective federal fund rate. The forecasting results show that all multiresolution-based prediction systems outperform the conventional reference models on the criteria of mean absolute error, mean absolute deviation, and root mean-squared error. Therefore, it is advantageous to adopt hybrid multiresolution techniques and soft computing models to forecast interest rate daily variations as they provide good forecasting performance.

A spiking neural network based cortex-like mechanism and application to facial expression recognition.

PubMed

Fu, Si-Yao; Yang, Guo-Sheng; Kuai, Xin-Kai

2012-01-01

In this paper, we present a quantitative, highly structured cortex-simulated model, which can be simply described as feedforward, hierarchical simulation of ventral stream of visual cortex using biologically plausible, computationally convenient spiking neural network system. The motivation comes directly from recent pioneering works on detailed functional decomposition analysis of the feedforward pathway of the ventral stream of visual cortex and developments on artificial spiking neural networks (SNNs). By combining the logical structure of the cortical hierarchy and computing power of the spiking neuron model, a practical framework has been presented. As a proof of principle, we demonstrate our system on several facial expression recognition tasks. The proposed cortical-like feedforward hierarchy framework has the merit of capability of dealing with complicated pattern recognition problems, suggesting that, by combining the cognitive models with modern neurocomputational approaches, the neurosystematic approach to the study of cortex-like mechanism has the potential to extend our knowledge of brain mechanisms underlying the cognitive analysis and to advance theoretical models of how we recognize face or, more specifically, perceive other people's facial expression in a rich, dynamic, and complex environment, providing a new starting point for improved models of visual cortex-like mechanism.

Comparing feed-forward versus neural gas as estimators: application to coke wastewater treatment.

PubMed

Machón-González, Iván; López-García, Hilario; Rodríguez-Iglesias, Jesús; Marañón-Maison, Elena; Castrillón-Peláez, Leonor; Fernández-Nava, Yolanda

2013-01-01

Numerous papers related to the estimation of wastewater parameters have used artificial neural networks. Although successful results have been reported, different problems have arisen such as overtraining, local minima and model instability. In this paper, two types of neural networks, feed-forward and neural gas, are trained to obtain a model that estimates the values of nitrates in the effluent stream of a three-step activated sludge system (two oxic and one anoxic). Placing the denitrification (anoxic) step at the head of the process can force denitrifying bacteria to use internal organic carbon. However, methanol has to be added to achieve high denitrification efficiencies in some industrial wastewaters. The aim of this paper is to compare the two networks in addition to suggesting a methodology to validate the models. The influence of the neighbourhood radius is important in the neural gas approach and must be selected correctly. Neural gas performs well due to its cooperation--competition procedure, with no problems of stability or overfitting arising in the experimental results. The neural gas model is also interesting for use as a direct plant model because of its robustness and deterministic behaviour.

Bringing Interpretability and Visualization with Artificial Neural Networks

ERIC Educational Resources Information Center

Gritsenko, Andrey

2017-01-01

Extreme Learning Machine (ELM) is a training algorithm for Single-Layer Feed-forward Neural Network (SLFN). The difference in theory of ELM from other training algorithms is in the existence of explicitly-given solution due to the immutability of initialed weights. In practice, ELMs achieve performance similar to that of other state-of-the-art…

Prediction of pelvic organ prolapse using an artificial neural network.

PubMed

Robinson, Christopher J; Swift, Steven; Johnson, Donna D; Almeida, Jonas S

2008-08-01

The objective of this investigation was to test the ability of a feedforward artificial neural network (ANN) to differentiate patients who have pelvic organ prolapse (POP) from those who retain good pelvic organ support. Following institutional review board approval, patients with POP (n = 87) and controls with good pelvic organ support (n = 368) were identified from the urogynecology research database. Historical and clinical information was extracted from the database. Data analysis included the training of a feedforward ANN, variable selection, and external validation of the model with an independent data set. Twenty variables were used. The median-performing ANN model used a median of 3 (quartile 1:3 to quartile 3:5) variables and achieved an area under the receiver operator curve of 0.90 (external, independent validation set). Ninety percent sensitivity and 83% specificity were obtained in the external validation by ANN classification. Feedforward ANN modeling is applicable to the identification and prediction of POP.

Center for Neural Engineering: applications of pulse-coupled neural networks

NASA Astrophysics Data System (ADS)

Malkani, Mohan; Bodruzzaman, Mohammad; Johnson, John L.; Davis, Joel

1999-03-01

Pulsed-Coupled Neural Network (PCNN) is an oscillatory model neural network where grouping of cells and grouping among the groups that form the output time series (number of cells that fires in each input presentation also called `icon'). This is based on the synchronicity of oscillations. Recent work by Johnson and others demonstrated the functional capabilities of networks containing such elements for invariant feature extraction using intensity maps. PCNN thus presents itself as a more biologically plausible model with solid functional potential. This paper will present the summary of several projects and their results where we successfully applied PCNN. In project one, the PCNN was applied for object recognition and classification through a robotic vision system. The features (icons) generated by the PCNN were then fed into a feedforward neural network for classification. In project two, we developed techniques for sensory data fusion. The PCNN algorithm was implemented and tested on a B14 mobile robot. The PCNN-based features were extracted from the images taken from the robot vision system and used in conjunction with the map generated by data fusion of the sonar and wheel encoder data for the navigation of the mobile robot. In our third project, we applied the PCNN for speaker recognition. The spectrogram image of speech signals are fed into the PCNN to produce invariant feature icons which are then fed into a feedforward neural network for speaker identification.

Artificial neural network modeling and optimization of ultrahigh pressure extraction of green tea polyphenols.

PubMed

Xi, Jun; Xue, Yujing; Xu, Yinxiang; Shen, Yuhong

2013-11-01

In this study, the ultrahigh pressure extraction of green tea polyphenols was modeled and optimized by a three-layer artificial neural network. A feed-forward neural network trained with an error back-propagation algorithm was used to evaluate the effects of pressure, liquid/solid ratio and ethanol concentration on the total phenolic content of green tea extracts. The neural network coupled with genetic algorithms was also used to optimize the conditions needed to obtain the highest yield of tea polyphenols. The obtained optimal architecture of artificial neural network model involved a feed-forward neural network with three input neurons, one hidden layer with eight neurons and one output layer including single neuron. The trained network gave the minimum value in the MSE of 0.03 and the maximum value in the R(2) of 0.9571, which implied a good agreement between the predicted value and the actual value, and confirmed a good generalization of the network. Based on the combination of neural network and genetic algorithms, the optimum extraction conditions for the highest yield of green tea polyphenols were determined as follows: 498.8 MPa for pressure, 20.8 mL/g for liquid/solid ratio and 53.6% for ethanol concentration. The total phenolic content of the actual measurement under the optimum predicated extraction conditions was 582.4 ± 0.63 mg/g DW, which was well matched with the predicted value (597.2mg/g DW). This suggests that the artificial neural network model described in this work is an efficient quantitative tool to predict the extraction efficiency of green tea polyphenols. Crown Copyright © 2013. Published by Elsevier Ltd. All rights reserved.

A training rule which guarantees finite-region stability for a class of closed-loop neural-network control systems.

PubMed

Kuntanapreeda, S; Fullmer, R R

1996-01-01

A training method for a class of neural network controllers is presented which guarantees closed-loop system stability. The controllers are assumed to be nonlinear, feedforward, sampled-data, full-state regulators implemented as single hidden-layer neural networks. The controlled systems must be locally hermitian and observable. Stability of the closed-loop system is demonstrated by determining a Lyapunov function, which can be used to identify a finite stability region about the regulator point.

Feed-Forward Neural Network Prediction of the Mechanical Properties of Sandcrete Materials

PubMed Central

Asteris, Panagiotis G.; Roussis, Panayiotis C.; Douvika, Maria G.

2017-01-01

This work presents a soft-sensor approach for estimating critical mechanical properties of sandcrete materials. Feed-forward (FF) artificial neural network (ANN) models are employed for building soft-sensors able to predict the 28-day compressive strength and the modulus of elasticity of sandcrete materials. To this end, a new normalization technique for the pre-processing of data is proposed. The comparison of the derived results with the available experimental data demonstrates the capability of FF ANNs to predict with pinpoint accuracy the mechanical properties of sandcrete materials. Furthermore, the proposed normalization technique has been proven effective and robust compared to other normalization techniques available in the literature. PMID:28598400

Artificial neural networks in Space Station optimal attitude control

NASA Astrophysics Data System (ADS)

Kumar, Renjith R.; Seywald, Hans; Deshpande, Samir M.; Rahman, Zia

1995-01-01

Innovative techniques of using "artificial neural networks" (ANN) for improving the performance of the pitch axis attitude control system of Space Station Freedom using control moment gyros (CMGs) are investigated. The first technique uses a feed-forward ANN with multi-layer perceptrons to obtain an on-line controller which improves the performance of the control system via a model following approach. The second technique uses a single layer feed-forward ANN with a modified back propagation scheme to estimate the internal plant variations and the external disturbances separately. These estimates are then used to solve two differential Riccati equations to obtain time varying gains which improve the control system performance in successive orbits.

Synaptic convergence regulates synchronization-dependent spike transfer in feedforward neural networks.

PubMed

Sailamul, Pachaya; Jang, Jaeson; Paik, Se-Bum

2017-12-01

Correlated neural activities such as synchronizations can significantly alter the characteristics of spike transfer between neural layers. However, it is not clear how this synchronization-dependent spike transfer can be affected by the structure of convergent feedforward wiring. To address this question, we implemented computer simulations of model neural networks: a source and a target layer connected with different types of convergent wiring rules. In the Gaussian-Gaussian (GG) model, both the connection probability and the strength are given as Gaussian distribution as a function of spatial distance. In the Uniform-Constant (UC) and Uniform-Exponential (UE) models, the connection probability density is a uniform constant within a certain range, but the connection strength is set as a constant value or an exponentially decaying function, respectively. Then we examined how the spike transfer function is modulated under these conditions, while static or synchronized input patterns were introduced to simulate different levels of feedforward spike synchronization. We observed that the synchronization-dependent modulation of the transfer function appeared noticeably different for each convergence condition. The modulation of the spike transfer function was largest in the UC model, and smallest in the UE model. Our analysis showed that this difference was induced by the different spike weight distributions that was generated from convergent synapses in each model. Our results suggest that, the structure of the feedforward convergence is a crucial factor for correlation-dependent spike control, thus must be considered important to understand the mechanism of information transfer in the brain.

Neural computation of arithmetic functions

NASA Technical Reports Server (NTRS)

Siu, Kai-Yeung; Bruck, Jehoshua

1990-01-01

An area of application of neural networks is considered. A neuron is modeled as a linear threshold gate, and the network architecture considered is the layered feedforward network. It is shown how common arithmetic functions such as multiplication and sorting can be efficiently computed in a shallow neural network. Some known results are improved by showing that the product of two n-bit numbers and sorting of n n-bit numbers can be computed by a polynomial-size neural network using only four and five unit delays, respectively. Moreover, the weights of each threshold element in the neural networks require O(log n)-bit (instead of n-bit) accuracy. These results can be extended to more complicated functions such as multiple products, division, rational functions, and approximation of analytic functions.

Neural Networks for Rapid Design and Analysis

NASA Technical Reports Server (NTRS)

Sparks, Dean W., Jr.; Maghami, Peiman G.

1998-01-01

Artificial neural networks have been employed for rapid and efficient dynamics and control analysis of flexible systems. Specifically, feedforward neural networks are designed to approximate nonlinear dynamic components over prescribed input ranges, and are used in simulations as a means to speed up the overall time response analysis process. To capture the recursive nature of dynamic components with artificial neural networks, recurrent networks, which use state feedback with the appropriate number of time delays, as inputs to the networks, are employed. Once properly trained, neural networks can give very good approximations to nonlinear dynamic components, and by their judicious use in simulations, allow the analyst the potential to speed up the analysis process considerably. To illustrate this potential speed up, an existing simulation model of a spacecraft reaction wheel system is executed, first conventionally, and then with an artificial neural network in place.

Classification of remotely sensed data using OCR-inspired neural network techniques. [Optical Character Recognition

NASA Technical Reports Server (NTRS)

Kiang, Richard K.

1992-01-01

Neural networks have been applied to classifications of remotely sensed data with some success. To improve the performance of this approach, an examination was made of how neural networks are applied to the optical character recognition (OCR) of handwritten digits and letters. A three-layer, feedforward network, along with techniques adopted from OCR, was used to classify Landsat-4 Thematic Mapper data. Good results were obtained. To overcome the difficulties that are characteristic of remote sensing applications and to attain significant improvements in classification accuracy, a special network architecture may be required.

A Prior for Neural Networks utilizing Enclosing Spheres for Normalization

NASA Astrophysics Data System (ADS)

v. Toussaint, U.; Gori, S.; Dose, V.

2004-11-01

Neural Networks are famous for their advantageous flexibility for problems when there is insufficient knowledge to set up a proper model. On the other hand this flexibility can cause over-fitting and can hamper the generalization properties of neural networks. Many approaches to regularize NN have been suggested but most of them based on ad-hoc arguments. Employing the principle of transformation invariance we derive a general prior in accordance with the Bayesian probability theory for a class of feedforward networks. Optimal networks are determined by Bayesian model comparison verifying the applicability of this approach.

Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting.

PubMed

Coop, Robert; Mishtal, Aaron; Arel, Itamar

2013-10-01

Catastrophic forgetting is a well-studied attribute of most parameterized supervised learning systems. A variation of this phenomenon, in the context of feedforward neural networks, arises when nonstationary inputs lead to loss of previously learned mappings. The majority of the schemes proposed in the literature for mitigating catastrophic forgetting were not data driven and did not scale well. We introduce the fixed expansion layer (FEL) feedforward neural network, which embeds a sparsely encoding hidden layer to help mitigate forgetting of prior learned representations. In addition, we investigate a novel framework for training ensembles of FEL networks, based on exploiting an information-theoretic measure of diversity between FEL learners, to further control undesired plasticity. The proposed methodology is demonstrated on a basic classification task, clearly emphasizing its advantages over existing techniques. The architecture proposed can be enhanced to address a range of computational intelligence tasks, such as regression problems and system control.

Functional approximation using artificial neural networks in structural mechanics

NASA Technical Reports Server (NTRS)

Alam, Javed; Berke, Laszlo

1993-01-01

The artificial neural networks (ANN) methodology is an outgrowth of research in artificial intelligence. In this study, the feed-forward network model that was proposed by Rumelhart, Hinton, and Williams was applied to the mapping of functions that are encountered in structural mechanics problems. Several different network configurations were chosen to train the available data for problems in materials characterization and structural analysis of plates and shells. By using the recall process, the accuracy of these trained networks was assessed.

Inverse kinematics problem in robotics using neural networks

NASA Technical Reports Server (NTRS)

Choi, Benjamin B.; Lawrence, Charles

1992-01-01

In this paper, Multilayer Feedforward Networks are applied to the robot inverse kinematic problem. The networks are trained with endeffector position and joint angles. After training, performance is measured by having the network generate joint angles for arbitrary endeffector trajectories. A 3-degree-of-freedom (DOF) spatial manipulator is used for the study. It is found that neural networks provide a simple and effective way to both model the manipulator inverse kinematics and circumvent the problems associated with algorithmic solution methods.

Neural network feedforward control of a closed-circuit wind tunnel

NASA Astrophysics Data System (ADS)

Sutcliffe, Peter

Accurate control of wind-tunnel test conditions can be dramatically enhanced using feedforward control architectures which allow operating conditions to be maintained at a desired setpoint through the use of mathematical models as the primary source of prediction. However, as the desired accuracy of the feedforward prediction increases, the model complexity also increases, so that an ever increasing computational load is incurred. This drawback can be avoided by employing a neural network that is trained offline using the output of a high fidelity wind-tunnel mathematical model, so that the neural network can rapidly reproduce the predictions of the model with a greatly reduced computational overhead. A novel neural network database generation method, developed through the use of fractional factorial arrays, was employed such that a neural network can accurately predict wind-tunnel parameters across a wide range of operating conditions whilst trained upon a highly efficient database. The subsequent network was incorporated into a Neural Network Model Predictive Control (NNMPC) framework to allow an optimised output schedule capable of providing accurate control of the wind-tunnel operating parameters. Facilitation of an optimised path through the solution space is achieved through the use of a chaos optimisation algorithm such that a more globally optimum solution is likely to be found with less computational expense than the gradient descent method. The parameters associated with the NNMPC such as the control horizon are determined through the use of a Taguchi methodology enabling the minimum number of experiments to be carried out to determine the optimal combination. The resultant NNMPC scheme was employed upon the Hessert Low Speed Wind Tunnel at the University of Notre Dame to control the test-section temperature such that it follows a pre-determined reference trajectory during changes in the test-section velocity. Experimental testing revealed that the derived NNMPC controller provided an excellent level of control over the test-section temperature in adherence to a reference trajectory even when faced with unforeseen disturbances such as rapid changes in the operating environment.

Nonlinear adaptive inverse control via the unified model neural network

NASA Astrophysics Data System (ADS)

Jeng, Jin-Tsong; Lee, Tsu-Tian

1999-03-01

In this paper, we propose a new nonlinear adaptive inverse control via a unified model neural network. In order to overcome nonsystematic design and long training time in nonlinear adaptive inverse control, we propose the approximate transformable technique to obtain a Chebyshev Polynomials Based Unified Model (CPBUM) neural network for the feedforward/recurrent neural networks. It turns out that the proposed method can use less training time to get an inverse model. Finally, we apply this proposed method to control magnetic bearing system. The experimental results show that the proposed nonlinear adaptive inverse control architecture provides a greater flexibility and better performance in controlling magnetic bearing systems.

Training feed-forward neural networks with gain constraints

PubMed

Hartman

2000-04-01

Inaccurate input-output gains (partial derivatives of outputs with respect to inputs) are common in neural network models when input variables are correlated or when data are incomplete or inaccurate. Accurate gains are essential for optimization, control, and other purposes. We develop and explore a method for training feedforward neural networks subject to inequality or equality-bound constraints on the gains of the learned mapping. Gain constraints are implemented as penalty terms added to the objective function, and training is done using gradient descent. Adaptive and robust procedures are devised for balancing the relative strengths of the various terms in the objective function, which is essential when the constraints are inconsistent with the data. The approach has the virtue that the model domain of validity can be extended via extrapolation training, which can dramatically improve generalization. The algorithm is demonstrated here on artificial and real-world problems with very good results and has been advantageously applied to dozens of models currently in commercial use.

Extraction of texture features with a multiresolution neural network

NASA Astrophysics Data System (ADS)

Lepage, Richard; Laurendeau, Denis; Gagnon, Roger A.

1992-09-01

Texture is an important surface characteristic. Many industrial materials such as wood, textile, or paper are best characterized by their texture. Detection of defaults occurring on such materials or classification for quality control anD matching can be carried out through careful texture analysis. A system for the classification of pieces of wood used in the furniture industry is proposed. This paper is concerned with a neural network implementation of the features extraction and classification components of the proposed system. Texture appears differently depending at which spatial scale it is observed. A complete description of a texture thus implies an analysis at several spatial scales. We propose a compact pyramidal representation of the input image for multiresolution analysis. The feature extraction system is implemented on a multilayer artificial neural network. Each level of the pyramid, which is a representation of the input image at a given spatial resolution scale, is mapped into a layer of the neural network. A full resolution texture image is input at the base of the pyramid and a representation of the texture image at multiple resolutions is generated by the feedforward pyramid structure of the neural network. The receptive field of each neuron at a given pyramid level is preprogrammed as a discrete Gaussian low-pass filter. Meaningful characteristics of the textured image must be extracted if a good resolving power of the classifier must be achieved. Local dominant orientation is the principal feature which is extracted from the textured image. Local edge orientation is computed with a Sobel mask at four orientation angles (multiple of (pi) /4). The resulting intrinsic image, that is, the local dominant orientation image, is fed to the texture classification neural network. The classification network is a three-layer feedforward back-propagation neural network.

Extracting grid cell characteristics from place cell inputs using non-negative principal component analysis

PubMed Central

Dordek, Yedidyah; Soudry, Daniel; Meir, Ron; Derdikman, Dori

2016-01-01

Many recent models study the downstream projection from grid cells to place cells, while recent data have pointed out the importance of the feedback projection. We thus asked how grid cells are affected by the nature of the input from the place cells. We propose a single-layer neural network with feedforward weights connecting place-like input cells to grid cell outputs. Place-to-grid weights are learned via a generalized Hebbian rule. The architecture of this network highly resembles neural networks used to perform Principal Component Analysis (PCA). Both numerical results and analytic considerations indicate that if the components of the feedforward neural network are non-negative, the output converges to a hexagonal lattice. Without the non-negativity constraint, the output converges to a square lattice. Consistent with experiments, grid spacing ratio between the first two consecutive modules is −1.4. Our results express a possible linkage between place cell to grid cell interactions and PCA. DOI: http://dx.doi.org/10.7554/eLife.10094.001 PMID:26952211

Optimization of Training Sets For Neural-Net Processing of Characteristic Patterns From Vibrating Solids

NASA Technical Reports Server (NTRS)

Decker, Arthur J. (Inventor)

2006-01-01

An artificial neural network is disclosed that processes holography generated characteristic pattern of vibrating structures along with finite-element models. The present invention provides for a folding operation for conditioning training sets for optimally training forward-neural networks to process characteristic fringe pattern. The folding pattern increases the sensitivity of the feed-forward network for detecting changes in the characteristic pattern The folding routine manipulates input pixels so as to be scaled according to the location in an intensity range rather than the position in the characteristic pattern.

Neural network-based run-to-run controller using exposure and resist thickness adjustment

NASA Astrophysics Data System (ADS)

Geary, Shane; Barry, Ronan

2003-06-01

This paper describes the development of a run-to-run control algorithm using a feedforward neural network, trained using the backpropagation training method. The algorithm is used to predict the critical dimension of the next lot using previous lot information. It is compared to a common prediction algorithm - the exponentially weighted moving average (EWMA) and is shown to give superior prediction performance in simulations. The manufacturing implementation of the final neural network showed significantly improved process capability when compared to the case where no run-to-run control was utilised.

Curriculum Assessment Using Artificial Neural Network and Support Vector Machine Modeling Approaches: A Case Study. IR Applications. Volume 29

ERIC Educational Resources Information Center

Chen, Chau-Kuang

2010-01-01

Artificial Neural Network (ANN) and Support Vector Machine (SVM) approaches have been on the cutting edge of science and technology for pattern recognition and data classification. In the ANN model, classification accuracy can be achieved by using the feed-forward of inputs, back-propagation of errors, and the adjustment of connection weights. In…

Learning a single-hidden layer feedforward neural network using a rank correlation-based strategy with application to high dimensional gene expression and proteomic spectra datasets in cancer detection.

PubMed

Belciug, Smaranda; Gorunescu, Florin

2018-06-08

Methods based on microarrays (MA), mass spectrometry (MS), and machine learning (ML) algorithms have evolved rapidly in recent years, allowing for early detection of several types of cancer. A pitfall of these approaches, however, is the overfitting of data due to large number of attributes and small number of instances -- a phenomenon known as the 'curse of dimensionality'. A potentially fruitful idea to avoid this drawback is to develop algorithms that combine fast computation with a filtering module for the attributes. The goal of this paper is to propose a statistical strategy to initiate the hidden nodes of a single-hidden layer feedforward neural network (SLFN) by using both the knowledge embedded in data and a filtering mechanism for attribute relevance. In order to attest its feasibility, the proposed model has been tested on five publicly available high-dimensional datasets: breast, lung, colon, and ovarian cancer regarding gene expression and proteomic spectra provided by cDNA arrays, DNA microarray, and MS. The novel algorithm, called adaptive SLFN (aSLFN), has been compared with four major classification algorithms: traditional ELM, radial basis function network (RBF), single-hidden layer feedforward neural network trained by backpropagation algorithm (BP-SLFN), and support vector-machine (SVM). Experimental results showed that the classification performance of aSLFN is competitive with the comparison models. Copyright © 2018. Published by Elsevier Inc.

Proceedings of the Government Neural Network Applications Workshop Held at Wright-Patterson AFB, Ohio on August 24-26, 1992. Volume 1

DTIC Science & Technology

1992-08-01

history trace of input u(t). (b) A common network struc- 1 ture makes use of the feedforward tapped delay line. For this structure the memory depth D...theories and analyses that will be used world- wide for a long time to come. The reason for this contribution has generally been the government’s need to...that emulate the neural reasoning behavior of biological neural systems (e.g. the human brain). As such, they are loosely based on biological neural

Sensitivity of feedforward neural networks to weight errors

NASA Technical Reports Server (NTRS)

Stevenson, Maryhelen; Widrow, Bernard; Winter, Rodney

1990-01-01

An analysis is made of the sensitivity of feedforward layered networks of Adaline elements (threshold logic units) to weight errors. An approximation is derived which expresses the probability of error for an output neuron of a large network (a network with many neurons per layer) as a function of the percentage change in the weights. As would be expected, the probability of error increases with the number of layers in the network and with the percentage change in the weights. The probability of error is essentially independent of the number of weights per neuron and of the number of neurons per layer, as long as these numbers are large (on the order of 100 or more).

A P2P Botnet detection scheme based on decision tree and adaptive multilayer neural networks.

PubMed

Alauthaman, Mohammad; Aslam, Nauman; Zhang, Li; Alasem, Rafe; Hossain, M A

2018-01-01

In recent years, Botnets have been adopted as a popular method to carry and spread many malicious codes on the Internet. These malicious codes pave the way to execute many fraudulent activities including spam mail, distributed denial-of-service attacks and click fraud. While many Botnets are set up using centralized communication architecture, the peer-to-peer (P2P) Botnets can adopt a decentralized architecture using an overlay network for exchanging command and control data making their detection even more difficult. This work presents a method of P2P Bot detection based on an adaptive multilayer feed-forward neural network in cooperation with decision trees. A classification and regression tree is applied as a feature selection technique to select relevant features. With these features, a multilayer feed-forward neural network training model is created using a resilient back-propagation learning algorithm. A comparison of feature set selection based on the decision tree, principal component analysis and the ReliefF algorithm indicated that the neural network model with features selection based on decision tree has a better identification accuracy along with lower rates of false positives. The usefulness of the proposed approach is demonstrated by conducting experiments on real network traffic datasets. In these experiments, an average detection rate of 99.08 % with false positive rate of 0.75 % was observed.

Theta phase precession and phase selectivity: a cognitive device description of neural coding

NASA Astrophysics Data System (ADS)

Zalay, Osbert C.; Bardakjian, Berj L.

2009-06-01

Information in neural systems is carried by way of phase and rate codes. Neuronal signals are processed through transformative biophysical mechanisms at the cellular and network levels. Neural coding transformations can be represented mathematically in a device called the cognitive rhythm generator (CRG). Incoming signals to the CRG are parsed through a bank of neuronal modes that orchestrate proportional, integrative and derivative transformations associated with neural coding. Mode outputs are then mixed through static nonlinearities to encode (spatio) temporal phase relationships. The static nonlinear outputs feed and modulate a ring device (limit cycle) encoding output dynamics. Small coupled CRG networks were created to investigate coding functionality associated with neuronal phase preference and theta precession in the hippocampus. Phase selectivity was found to be dependent on mode shape and polarity, while phase precession was a product of modal mixing (i.e. changes in the relative contribution or amplitude of mode outputs resulted in shifting phase preference). Nonlinear system identification was implemented to help validate the model and explain response characteristics associated with modal mixing; in particular, principal dynamic modes experimentally derived from a hippocampal neuron were inserted into a CRG and the neuron's dynamic response was successfully cloned. From our results, small CRG networks possessing disynaptic feedforward inhibition in combination with feedforward excitation exhibited frequency-dependent inhibitory-to-excitatory and excitatory-to-inhibitory transitions that were similar to transitions seen in a single CRG with quadratic modal mixing. This suggests nonlinear modal mixing to be a coding manifestation of the effect of network connectivity in shaping system dynamic behavior. We hypothesize that circuits containing disynaptic feedforward inhibition in the nervous system may be candidates for interpreting upstream rate codes to guide downstream processes such as phase precession, because of their demonstrated frequency-selective properties.

A quantum-implementable neural network model

NASA Astrophysics Data System (ADS)

Chen, Jialin; Wang, Lingli; Charbon, Edoardo

2017-10-01

A quantum-implementable neural network, namely quantum probability neural network (QPNN) model, is proposed in this paper. QPNN can use quantum parallelism to trace all possible network states to improve the result. Due to its unique quantum nature, this model is robust to several quantum noises under certain conditions, which can be efficiently implemented by the qubus quantum computer. Another advantage is that QPNN can be used as memory to retrieve the most relevant data and even to generate new data. The MATLAB experimental results of Iris data classification and MNIST handwriting recognition show that much less neuron resources are required in QPNN to obtain a good result than the classical feedforward neural network. The proposed QPNN model indicates that quantum effects are useful for real-life classification tasks.

Training Feedforward Neural Networks Using Symbiotic Organisms Search Algorithm.

PubMed

Wu, Haizhou; Zhou, Yongquan; Luo, Qifang; Basset, Mohamed Abdel

2016-01-01

Symbiotic organisms search (SOS) is a new robust and powerful metaheuristic algorithm, which stimulates the symbiotic interaction strategies adopted by organisms to survive and propagate in the ecosystem. In the supervised learning area, it is a challenging task to present a satisfactory and efficient training algorithm for feedforward neural networks (FNNs). In this paper, SOS is employed as a new method for training FNNs. To investigate the performance of the aforementioned method, eight different datasets selected from the UCI machine learning repository are employed for experiment and the results are compared among seven metaheuristic algorithms. The results show that SOS performs better than other algorithms for training FNNs in terms of converging speed. It is also proven that an FNN trained by the method of SOS has better accuracy than most algorithms compared.

A novel memristive multilayer feedforward small-world neural network with its applications in PID control.

PubMed

Dong, Zhekang; Duan, Shukai; Hu, Xiaofang; Wang, Lidan; Li, Hai

2014-01-01

In this paper, we present an implementation scheme of memristor-based multilayer feedforward small-world neural network (MFSNN) inspirited by the lack of the hardware realization of the MFSNN on account of the need of a large number of electronic neurons and synapses. More specially, a mathematical closed-form charge-governed memristor model is presented with derivation procedures and the corresponding Simulink model is presented, which is an essential block for realizing the memristive synapse and the activation function in electronic neurons. Furthermore, we investigate a more intelligent memristive PID controller by incorporating the proposed MFSNN into intelligent PID control based on the advantages of the memristive MFSNN on computation speed and accuracy. Finally, numerical simulations have demonstrated the effectiveness of the proposed scheme.

A Novel Memristive Multilayer Feedforward Small-World Neural Network with Its Applications in PID Control

PubMed Central

Dong, Zhekang; Duan, Shukai; Hu, Xiaofang; Wang, Lidan

2014-01-01

In this paper, we present an implementation scheme of memristor-based multilayer feedforward small-world neural network (MFSNN) inspirited by the lack of the hardware realization of the MFSNN on account of the need of a large number of electronic neurons and synapses. More specially, a mathematical closed-form charge-governed memristor model is presented with derivation procedures and the corresponding Simulink model is presented, which is an essential block for realizing the memristive synapse and the activation function in electronic neurons. Furthermore, we investigate a more intelligent memristive PID controller by incorporating the proposed MFSNN into intelligent PID control based on the advantages of the memristive MFSNN on computation speed and accuracy. Finally, numerical simulations have demonstrated the effectiveness of the proposed scheme. PMID:25202723

Analysis and prediction of aperiodic hydrodynamic oscillatory time series by feed-forward neural networks, fuzzy logic, and a local nonlinear predictor

DOE Office of Scientific and Technical Information (OSTI.GOV)

Gentili, Pier Luigi, E-mail: pierluigi.gentili@unipg.it; Gotoda, Hiroshi; Dolnik, Milos

Forecasting of aperiodic time series is a compelling challenge for science. In this work, we analyze aperiodic spectrophotometric data, proportional to the concentrations of two forms of a thermoreversible photochromic spiro-oxazine, that are generated when a cuvette containing a solution of the spiro-oxazine undergoes photoreaction and convection due to localized ultraviolet illumination. We construct the phase space for the system using Takens' theorem and we calculate the Lyapunov exponents and the correlation dimensions to ascertain the chaotic character of the time series. Finally, we predict the time series using three distinct methods: a feed-forward neural network, fuzzy logic, and amore » local nonlinear predictor. We compare the performances of these three methods.« less

A neural network architecture for implementation of expert systems for real time monitoring

NASA Technical Reports Server (NTRS)

Ramamoorthy, P. A.

1991-01-01

Since neural networks have the advantages of massive parallelism and simple architecture, they are good tools for implementing real time expert systems. In a rule based expert system, the antecedents of rules are in the conjunctive or disjunctive form. We constructed a multilayer feedforward type network in which neurons represent AND or OR operations of rules. Further, we developed a translator which can automatically map a given rule base into the network. Also, we proposed a new and powerful yet flexible architecture that combines the advantages of both fuzzy expert systems and neural networks. This architecture uses the fuzzy logic concepts to separate input data domains into several smaller and overlapped regions. Rule-based expert systems for time critical applications using neural networks, the automated implementation of rule-based expert systems with neural nets, and fuzzy expert systems vs. neural nets are covered.

Neural network-based model reference adaptive control system.

PubMed

Patino, H D; Liu, D

2000-01-01

In this paper, an approach to model reference adaptive control based on neural networks is proposed and analyzed for a class of first-order continuous-time nonlinear dynamical systems. The controller structure can employ either a radial basis function network or a feedforward neural network to compensate adaptively the nonlinearities in the plant. A stable controller-parameter adjustment mechanism, which is determined using the Lyapunov theory, is constructed using a sigma-modification-type updating law. The evaluation of control error in terms of the neural network learning error is performed. That is, the control error converges asymptotically to a neighborhood of zero, whose size is evaluated and depends on the approximation error of the neural network. In the design and analysis of neural network-based control systems, it is important to take into account the neural network learning error and its influence on the control error of the plant. Simulation results showing the feasibility and performance of the proposed approach are given.

Application of neural networks to prediction of advanced composite structures mechanical response and behavior

NASA Technical Reports Server (NTRS)

Cios, K. J.; Vary, A.; Berke, L.; Kautz, H. E.

1992-01-01

Two types of neural networks were used to evaluate acousto-ultrasonic (AU) data for material characterization and mechanical reponse prediction. The neural networks included a simple feedforward network (backpropagation) and a radial basis functions network. Comparisons of results in terms of accuracy and training time are given. Acousto-ultrasonic (AU) measurements were performed on a series of tensile specimens composed of eight laminated layers of continuous, SiC fiber reinforced Ti-15-3 matrix. The frequency spectrum was dominated by frequencies of longitudinal wave resonance through the thickness of the specimen at the sending transducer. The magnitude of the frequency spectrum of the AU signal was used for calculating a stress-wave factor based on integrating the spectral distribution function and used for comparison with neural networks results.

Dynamic Pricing in Electronic Commerce Using Neural Network

NASA Astrophysics Data System (ADS)

Ghose, Tapu Kumar; Tran, Thomas T.

In this paper, we propose an approach where feed-forward neural network is used for dynamically calculating a competitive price of a product in order to maximize sellers’ revenue. In the approach we considered that along with product price other attributes such as product quality, delivery time, after sales service and seller’s reputation contribute in consumers purchase decision. We showed that once the sellers, by using their limited prior knowledge, set an initial price of a product our model adjusts the price automatically with the help of neural network so that sellers’ revenue is maximized.

Particle identification with neural networks using a rotational invariant moment representation

NASA Astrophysics Data System (ADS)

Sinkus, R.; Voss, T.

1997-02-01

A feed-forward neural network is used to identify electromagnetic particles based upon their showering properties within a segmented calorimeter. The novel feature is the expansion of the energy distribution in terms of moments of the so-called Zernike functions which are invariant under rotation. The multidimensional input distribution for the neural network is transformed via a principle component analysis and rescaled by its respective variances to ensure input values of the order of one. This results is a better performance in identifying and separating electromagnetic from hadronic particles, especially at low energies.

Prior-knowledge-based feedforward network simulation of true boiling point curve of crude oil.

PubMed

Chen, C W; Chen, D Z

2001-11-01

Theoretical results and practical experience indicate that feedforward networks can approximate a wide class of functional relationships very well. This property is exploited in modeling chemical processes. Given finite and noisy training data, it is important to encode the prior knowledge in neural networks to improve the fit precision and the prediction ability of the model. In this paper, as to the three-layer feedforward networks and the monotonic constraint, the unconstrained method, Joerding's penalty function method, the interpolation method, and the constrained optimization method are analyzed first. Then two novel methods, the exponential weight method and the adaptive method, are proposed. These methods are applied in simulating the true boiling point curve of a crude oil with the condition of increasing monotonicity. The simulation experimental results show that the network models trained by the novel methods are good at approximating the actual process. Finally, all these methods are discussed and compared with each other.

Linear and nonlinear ARMA model parameter estimation using an artificial neural network

NASA Technical Reports Server (NTRS)

Chon, K. H.; Cohen, R. J.

1997-01-01

This paper addresses parametric system identification of linear and nonlinear dynamic systems by analysis of the input and output signals. Specifically, we investigate the relationship between estimation of the system using a feedforward neural network model and estimation of the system by use of linear and nonlinear autoregressive moving-average (ARMA) models. By utilizing a neural network model incorporating a polynomial activation function, we show the equivalence of the artificial neural network to the linear and nonlinear ARMA models. We compare the parameterization of the estimated system using the neural network and ARMA approaches by utilizing data generated by means of computer simulations. Specifically, we show that the parameters of a simulated ARMA system can be obtained from the neural network analysis of the simulated data or by conventional least squares ARMA analysis. The feasibility of applying neural networks with polynomial activation functions to the analysis of experimental data is explored by application to measurements of heart rate (HR) and instantaneous lung volume (ILV) fluctuations.

An intelligent control system for failure detection and controller reconfiguration

NASA Technical Reports Server (NTRS)

Biswas, Saroj K.

1994-01-01

We present an architecture of an intelligent restructurable control system to automatically detect failure of system components, assess its impact on system performance and safety, and reconfigure the controller for performance recovery. Fault detection is based on neural network associative memories and pattern classifiers, and is implemented using a multilayer feedforward network. Details of the fault detection network along with simulation results on health monitoring of a dc motor have been presented. Conceptual developments for fault assessment using an expert system and controller reconfiguration using a neural network are outlined.

Biomimetic Hybrid Feedback Feedforward Neural-Network Learning Control.

PubMed

Pan, Yongping; Yu, Haoyong

2017-06-01

This brief presents a biomimetic hybrid feedback feedforward neural-network learning control (NNLC) strategy inspired by the human motor learning control mechanism for a class of uncertain nonlinear systems. The control structure includes a proportional-derivative controller acting as a feedback servo machine and a radial-basis-function (RBF) NN acting as a feedforward predictive machine. Under the sufficient constraints on control parameters, the closed-loop system achieves semiglobal practical exponential stability, such that an accurate NN approximation is guaranteed in a local region along recurrent reference trajectories. Compared with the existing NNLC methods, the novelties of the proposed method include: 1) the implementation of an adaptive NN control to guarantee plant states being recurrent is not needed, since recurrent reference signals rather than plant states are utilized as NN inputs, which greatly simplifies the analysis and synthesis of the NNLC and 2) the domain of NN approximation can be determined a priori by the given reference signals, which leads to an easy construction of the RBF-NNs. Simulation results have verified the effectiveness of this approach.

Multitask neurovision processor with extensive feedback and feedforward connections

NASA Astrophysics Data System (ADS)

Gupta, Madan M.; Knopf, George K.

1991-11-01

A multi-task neuro-vision parameter which performs a variety of information processing operations associated with the early stages of biological vision is presented. The network architecture of this neuro-vision processor, called the positive-negative (PN) neural processor, is loosely based on the neural activity fields exhibited by thalamic and cortical nervous tissue layers. The computational operation performed by the processor arises from the strength of the recurrent feedback among the numerous positive and negative neural computing units. By adjusting the feedback connections it is possible to generate diverse dynamic behavior that may be used for short-term visual memory (STVM), spatio-temporal filtering (STF), and pulse frequency modulation (PFM). The information attributes that are to be processes may be regulated by modifying the feedforward connections from the signal space to the neural processor.

A neural-network-based model for the dynamic simulation of the tire/suspension system while traversing road irregularities.

PubMed

Guarneri, Paolo; Rocca, Gianpiero; Gobbi, Massimiliano

2008-09-01

This paper deals with the simulation of the tire/suspension dynamics by using recurrent neural networks (RNNs). RNNs are derived from the multilayer feedforward neural networks, by adding feedback connections between output and input layers. The optimal network architecture derives from a parametric analysis based on the optimal tradeoff between network accuracy and size. The neural network can be trained with experimental data obtained in the laboratory from simulated road profiles (cleats). The results obtained from the neural network demonstrate good agreement with the experimental results over a wide range of operation conditions. The NN model can be effectively applied as a part of vehicle system model to accurately predict elastic bushings and tire dynamics behavior. Although the neural network model, as a black-box model, does not provide a good insight of the physical behavior of the tire/suspension system, it is a useful tool for assessing vehicle ride and noise, vibration, harshness (NVH) performance due to its good computational efficiency and accuracy.

Analog hardware for learning neural networks

NASA Technical Reports Server (NTRS)

Eberhardt, Silvio P. (Inventor)

1991-01-01

This is a recurrent or feedforward analog neural network processor having a multi-level neuron array and a synaptic matrix for storing weighted analog values of synaptic connection strengths which is characterized by temporarily changing one connection strength at a time to determine its effect on system output relative to the desired target. That connection strength is then adjusted based on the effect, whereby the processor is taught the correct response to training examples connection by connection.

Network evolution induced by asynchronous stimuli through spike-timing-dependent plasticity.

PubMed

Yuan, Wu-Jie; Zhou, Jian-Fang; Zhou, Changsong

2013-01-01

In sensory neural system, external asynchronous stimuli play an important role in perceptual learning, associative memory and map development. However, the organization of structure and dynamics of neural networks induced by external asynchronous stimuli are not well understood. Spike-timing-dependent plasticity (STDP) is a typical synaptic plasticity that has been extensively found in the sensory systems and that has received much theoretical attention. This synaptic plasticity is highly sensitive to correlations between pre- and postsynaptic firings. Thus, STDP is expected to play an important role in response to external asynchronous stimuli, which can induce segregative pre- and postsynaptic firings. In this paper, we study the impact of external asynchronous stimuli on the organization of structure and dynamics of neural networks through STDP. We construct a two-dimensional spatial neural network model with local connectivity and sparseness, and use external currents to stimulate alternately on different spatial layers. The adopted external currents imposed alternately on spatial layers can be here regarded as external asynchronous stimuli. Through extensive numerical simulations, we focus on the effects of stimulus number and inter-stimulus timing on synaptic connecting weights and the property of propagation dynamics in the resulting network structure. Interestingly, the resulting feedforward structure induced by stimulus-dependent asynchronous firings and its propagation dynamics reflect both the underlying property of STDP. The results imply a possible important role of STDP in generating feedforward structure and collective propagation activity required for experience-dependent map plasticity in developing in vivo sensory pathways and cortices. The relevance of the results to cue-triggered recall of learned temporal sequences, an important cognitive function, is briefly discussed as well. Furthermore, this finding suggests a potential application for examining STDP by measuring neural population activity in a cultured neural network.

Neural control and transient analysis of the LCL-type resonant converter

NASA Astrophysics Data System (ADS)

Zouggar, S.; Nait Charif, H.; Azizi, M.

2000-07-01

This paper proposes a generalised inverse learning structure to control the LCL converter. A feedforward neural network is trained to act as an inverse model of the LCL converter then both are cascaded such that the composed system results in an identity mapping between desired response and the LCL output voltage. Using the large signal model, we analyse the transient output response of the controlled LCL converter in the case of large variation of the load. The simulation results show the efficiency of using neural networks to regulate the LCL converter.

Training Feedforward Neural Networks Using Symbiotic Organisms Search Algorithm

PubMed Central

Wu, Haizhou; Luo, Qifang

2016-01-01

Symbiotic organisms search (SOS) is a new robust and powerful metaheuristic algorithm, which stimulates the symbiotic interaction strategies adopted by organisms to survive and propagate in the ecosystem. In the supervised learning area, it is a challenging task to present a satisfactory and efficient training algorithm for feedforward neural networks (FNNs). In this paper, SOS is employed as a new method for training FNNs. To investigate the performance of the aforementioned method, eight different datasets selected from the UCI machine learning repository are employed for experiment and the results are compared among seven metaheuristic algorithms. The results show that SOS performs better than other algorithms for training FNNs in terms of converging speed. It is also proven that an FNN trained by the method of SOS has better accuracy than most algorithms compared. PMID:28105044

Redistribution of neural phase coherence reflects establishment of feedforward map in speech motor adaptation

PubMed Central

Sengupta, Ranit

2015-01-01

Despite recent progress in our understanding of sensorimotor integration in speech learning, a comprehensive framework to investigate its neural basis is lacking at behaviorally relevant timescales. Structural and functional imaging studies in humans have helped us identify brain networks that support speech but fail to capture the precise spatiotemporal coordination within the networks that takes place during speech learning. Here we use neuronal oscillations to investigate interactions within speech motor networks in a paradigm of speech motor adaptation under altered feedback with continuous recording of EEG in which subjects adapted to the real-time auditory perturbation of a target vowel sound. As subjects adapted to the task, concurrent changes were observed in the theta-gamma phase coherence during speech planning at several distinct scalp regions that is consistent with the establishment of a feedforward map. In particular, there was an increase in coherence over the central region and a decrease over the fronto-temporal regions, revealing a redistribution of coherence over an interacting network of brain regions that could be a general feature of error-based motor learning in general. Our findings have implications for understanding the neural basis of speech motor learning and could elucidate how transient breakdown of neuronal communication within speech networks relates to speech disorders. PMID:25632078

Neural networks within multi-core optic fibers

PubMed Central

Cohen, Eyal; Malka, Dror; Shemer, Amir; Shahmoon, Asaf; Zalevsky, Zeev; London, Michael

2016-01-01

Hardware implementation of artificial neural networks facilitates real-time parallel processing of massive data sets. Optical neural networks offer low-volume 3D connectivity together with large bandwidth and minimal heat production in contrast to electronic implementation. Here, we present a conceptual design for in-fiber optical neural networks. Neurons and synapses are realized as individual silica cores in a multi-core fiber. Optical signals are transferred transversely between cores by means of optical coupling. Pump driven amplification in erbium-doped cores mimics synaptic interactions. We simulated three-layered feed-forward neural networks and explored their capabilities. Simulations suggest that networks can differentiate between given inputs depending on specific configurations of amplification; this implies classification and learning capabilities. Finally, we tested experimentally our basic neuronal elements using fibers, couplers, and amplifiers, and demonstrated that this configuration implements a neuron-like function. Therefore, devices similar to our proposed multi-core fiber could potentially serve as building blocks for future large-scale small-volume optical artificial neural networks. PMID:27383911

Neural networks within multi-core optic fibers.

PubMed

Cohen, Eyal; Malka, Dror; Shemer, Amir; Shahmoon, Asaf; Zalevsky, Zeev; London, Michael

2016-07-07

Hardware implementation of artificial neural networks facilitates real-time parallel processing of massive data sets. Optical neural networks offer low-volume 3D connectivity together with large bandwidth and minimal heat production in contrast to electronic implementation. Here, we present a conceptual design for in-fiber optical neural networks. Neurons and synapses are realized as individual silica cores in a multi-core fiber. Optical signals are transferred transversely between cores by means of optical coupling. Pump driven amplification in erbium-doped cores mimics synaptic interactions. We simulated three-layered feed-forward neural networks and explored their capabilities. Simulations suggest that networks can differentiate between given inputs depending on specific configurations of amplification; this implies classification and learning capabilities. Finally, we tested experimentally our basic neuronal elements using fibers, couplers, and amplifiers, and demonstrated that this configuration implements a neuron-like function. Therefore, devices similar to our proposed multi-core fiber could potentially serve as building blocks for future large-scale small-volume optical artificial neural networks.

Rule extraction from minimal neural networks for credit card screening.

PubMed

Setiono, Rudy; Baesens, Bart; Mues, Christophe

2011-08-01

While feedforward neural networks have been widely accepted as effective tools for solving classification problems, the issue of finding the best network architecture remains unresolved, particularly so in real-world problem settings. We address this issue in the context of credit card screening, where it is important to not only find a neural network with good predictive performance but also one that facilitates a clear explanation of how it produces its predictions. We show that minimal neural networks with as few as one hidden unit provide good predictive accuracy, while having the added advantage of making it easier to generate concise and comprehensible classification rules for the user. To further reduce model size, a novel approach is suggested in which network connections from the input units to this hidden unit are removed by a very straightaway pruning procedure. In terms of predictive accuracy, both the minimized neural networks and the rule sets generated from them are shown to compare favorably with other neural network based classifiers. The rules generated from the minimized neural networks are concise and thus easier to validate in a real-life setting.

Multi-Layered Feedforward Neural Networks for Image Segmentation

DTIC Science & Technology

1991-12-01

the Gram-Schmidt Network ...................... 80 xi Preface WILLIAM SHAKESPEARE 1564-1616 Is this a dagger which I see before me, The handle toward...any input-output mapping with a single hidden layer of non-linear nodes, the result may be like proving that a monkey could write Hamlet . Certainly it

Valid approximation of spatially distributed grain size distributions - A priori information encoded to a feedforward network

NASA Astrophysics Data System (ADS)

Berthold, T.; Milbradt, P.; Berkhahn, V.

2018-04-01

This paper presents a model for the approximation of multiple, spatially distributed grain size distributions based on a feedforward neural network. Since a classical feedforward network does not guarantee to produce valid cumulative distribution functions, a priori information is incor porated into the model by applying weight and architecture constraints. The model is derived in two steps. First, a model is presented that is able to produce a valid distribution function for a single sediment sample. Although initially developed for sediment samples, the model is not limited in its application; it can also be used to approximate any other multimodal continuous distribution function. In the second part, the network is extended in order to capture the spatial variation of the sediment samples that have been obtained from 48 locations in the investigation area. Results show that the model provides an adequate approximation of grain size distributions, satisfying the requirements of a cumulative distribution function.

A Novel Approach to Noise-Filtering Based on a Gain-Scheduling Neural Network Architecture

NASA Technical Reports Server (NTRS)

Troudet, T.; Merrill, W.

1994-01-01

A gain-scheduling neural network architecture is proposed to enhance the noise-filtering efficiency of feedforward neural networks, in terms of both nominal performance and robustness. The synergistic benefits of the proposed architecture are demonstrated and discussed in the context of the noise-filtering of signals that are typically encountered in aerospace control systems. The synthesis of such a gain-scheduled neurofiltering provides the robustness of linear filtering, while preserving the nominal performance advantage of conventional nonlinear neurofiltering. Quantitative performance and robustness evaluations are provided for the signal processing of pitch rate responses to typical pilot command inputs for a modern fighter aircraft model.

Detection of pseudosinusoidal epileptic seizure segments in the neonatal EEG by cascading a rule-based algorithm with a neural network.

PubMed

Karayiannis, Nicolaos B; Mukherjee, Amit; Glover, John R; Ktonas, Periklis Y; Frost, James D; Hrachovy, Richard A; Mizrahi, Eli M

2006-04-01

This paper presents an approach to detect epileptic seizure segments in the neonatal electroencephalogram (EEG) by characterizing the spectral features of the EEG waveform using a rule-based algorithm cascaded with a neural network. A rule-based algorithm screens out short segments of pseudosinusoidal EEG patterns as epileptic based on features in the power spectrum. The output of the rule-based algorithm is used to train and compare the performance of conventional feedforward neural networks and quantum neural networks. The results indicate that the trained neural networks, cascaded with the rule-based algorithm, improved the performance of the rule-based algorithm acting by itself. The evaluation of the proposed cascaded scheme for the detection of pseudosinusoidal seizure segments reveals its potential as a building block of the automated seizure detection system under development.

Learning representations for the early detection of sepsis with deep neural networks.

PubMed

Kam, Hye Jin; Kim, Ha Young

2017-10-01

Sepsis is one of the leading causes of death in intensive care unit patients. Early detection of sepsis is vital because mortality increases as the sepsis stage worsens. This study aimed to develop detection models for the early stage of sepsis using deep learning methodologies, and to compare the feasibility and performance of the new deep learning methodology with those of the regression method with conventional temporal feature extraction. Study group selection adhered to the InSight model. The results of the deep learning-based models and the InSight model were compared. With deep feedforward networks, the area under the ROC curve (AUC) of the models were 0.887 and 0.915 for the InSight and the new feature sets, respectively. For the model with the combined feature set, the AUC was the same as that of the basic feature set (0.915). For the long short-term memory model, only the basic feature set was applied and the AUC improved to 0.929 compared with the existing 0.887 of the InSight model. The contributions of this paper can be summarized in three ways: (i) improved performance without feature extraction using domain knowledge, (ii) verification of feature extraction capability of deep neural networks through comparison with reference features, and (iii) improved performance with feedforward neural networks using long short-term memory, a neural network architecture that can learn sequential patterns. Copyright © 2017 Elsevier Ltd. All rights reserved.

Salient Feature Selection Using Feed-Forward Neural Networks and Signal-to-Noise Ratios with a Focus Toward Network Threat Detection and Classification

DTIC Science & Technology

2014-03-27

0.8.0. The virtual machine’s network adapter was set to internal network only to keep any outside traffic from interfering. A MySQL -based query...primary output of Fullstats is the ARFF file format, intended for use with the WEKA Java -based data mining software developed at the University of Waikato

Synthesis of recurrent neural networks for dynamical system simulation.

PubMed

Trischler, Adam P; D'Eleuterio, Gabriele M T

2016-08-01

We review several of the most widely used techniques for training recurrent neural networks to approximate dynamical systems, then describe a novel algorithm for this task. The algorithm is based on an earlier theoretical result that guarantees the quality of the network approximation. We show that a feedforward neural network can be trained on the vector-field representation of a given dynamical system using backpropagation, then recast it as a recurrent network that replicates the original system's dynamics. After detailing this algorithm and its relation to earlier approaches, we present numerical examples that demonstrate its capabilities. One of the distinguishing features of our approach is that both the original dynamical systems and the recurrent networks that simulate them operate in continuous time. Copyright © 2016 Elsevier Ltd. All rights reserved.

Artificial neural networks for processing fluorescence spectroscopy data in skin cancer diagnostics

NASA Astrophysics Data System (ADS)

Lenhardt, L.; Zeković, I.; Dramićanin, T.; Dramićanin, M. D.

2013-11-01

Over the years various optical spectroscopic techniques have been widely used as diagnostic tools in the discrimination of many types of malignant diseases. Recently, synchronous fluorescent spectroscopy (SFS) coupled with chemometrics has been applied in cancer diagnostics. The SFS method involves simultaneous scanning of both emission and excitation wavelengths while keeping the interval of wavelengths (constant-wavelength mode) or frequencies (constant-energy mode) between them constant. This method is fast, relatively inexpensive, sensitive and non-invasive. Total synchronous fluorescence spectra of normal skin, nevus and melanoma samples were used as input for training of artificial neural networks. Two different types of artificial neural networks were trained, the self-organizing map and the feed-forward neural network. Histopathology results of investigated skin samples were used as the gold standard for network output. Based on the obtained classification success rate of neural networks, we concluded that both networks provided high sensitivity with classification errors between 2 and 4%.

Prediction of friction factor of pure water flowing inside vertical smooth and microfin tubes by using artificial neural networks

NASA Astrophysics Data System (ADS)

Çebi, A.; Akdoğan, E.; Celen, A.; Dalkilic, A. S.

2017-02-01

An artificial neural network (ANN) model of friction factor in smooth and microfin tubes under heating, cooling and isothermal conditions was developed in this study. Data used in ANN was taken from a vertically positioned heat exchanger experimental setup. Multi-layered feed-forward neural network with backpropagation algorithm, radial basis function networks and hybrid PSO-neural network algorithm were applied to the database. Inputs were the ratio of cross sectional flow area to hydraulic diameter, experimental condition number depending on isothermal, heating, or cooling conditions and mass flow rate while the friction factor was the output of the constructed system. It was observed that such neural network based system could effectively predict the friction factor values of the flows regardless of their tube types. A dependency analysis to determine the strongest parameter that affected the network and database was also performed and tube geometry was found to be the strongest parameter of all as a result of analysis.

Feed-Forward Neural Network Soft-Sensor Modeling of Flotation Process Based on Particle Swarm Optimization and Gravitational Search Algorithm

PubMed Central

Wang, Jie-Sheng; Han, Shuang

2015-01-01

For predicting the key technology indicators (concentrate grade and tailings recovery rate) of flotation process, a feed-forward neural network (FNN) based soft-sensor model optimized by the hybrid algorithm combining particle swarm optimization (PSO) algorithm and gravitational search algorithm (GSA) is proposed. Although GSA has better optimization capability, it has slow convergence velocity and is easy to fall into local optimum. So in this paper, the velocity vector and position vector of GSA are adjusted by PSO algorithm in order to improve its convergence speed and prediction accuracy. Finally, the proposed hybrid algorithm is adopted to optimize the parameters of FNN soft-sensor model. Simulation results show that the model has better generalization and prediction accuracy for the concentrate grade and tailings recovery rate to meet the online soft-sensor requirements of the real-time control in the flotation process. PMID:26583034

Detection of different states of sleep in the rodents by the means of artificial neural networks

NASA Astrophysics Data System (ADS)

Musatov, Viacheslav; Dykin, Viacheslav; Pitsik, Elena; Pisarchik, Alexander

2018-04-01

This paper considers the possibility of classification of electroencephalogram (EEG) and electromyogram (EMG) signals corresponding to different phases of sleep and wakefulness of mice by the means of artificial neural networks. A feed-forward artificial neural network based on multilayer perceptron was created and trained on the data of one of the rodents. The trained network was used to read and classify the EEG and EMG data corresponding to different phases of sleep and wakefulness of the same mouse and other mouse. The results show a good recognition quality of all phases for the rodent on which the training was conducted (80-99%) and acceptable recognition quality for the data collected from the same mouse after a stroke.

Processing speed in recurrent visual networks correlates with general intelligence.

PubMed

Jolij, Jacob; Huisman, Danielle; Scholte, Steven; Hamel, Ronald; Kemner, Chantal; Lamme, Victor A F

2007-01-08

Studies on the neural basis of general fluid intelligence strongly suggest that a smarter brain processes information faster. Different brain areas, however, are interconnected by both feedforward and feedback projections. Whether both types of connections or only one of the two types are faster in smarter brains remains unclear. Here we show, by measuring visual evoked potentials during a texture discrimination task, that general fluid intelligence shows a strong correlation with processing speed in recurrent visual networks, while there is no correlation with speed of feedforward connections. The hypothesis that a smarter brain runs faster may need to be refined: a smarter brain's feedback connections run faster.

Neural networks for tracking of unknown SISO discrete-time nonlinear dynamic systems.

PubMed

Aftab, Muhammad Saleheen; Shafiq, Muhammad

2015-11-01

This article presents a Lyapunov function based neural network tracking (LNT) strategy for single-input, single-output (SISO) discrete-time nonlinear dynamic systems. The proposed LNT architecture is composed of two feedforward neural networks operating as controller and estimator. A Lyapunov function based back propagation learning algorithm is used for online adjustment of the controller and estimator parameters. The controller and estimator error convergence and closed-loop system stability analysis is performed by Lyapunov stability theory. Moreover, two simulation examples and one real-time experiment are investigated as case studies. The achieved results successfully validate the controller performance. Copyright © 2015 ISA. Published by Elsevier Ltd. All rights reserved.

Design of order statistics filters using feedforward neural networks

NASA Astrophysics Data System (ADS)

Maslennikova, Yu. S.; Bochkarev, V. V.

2016-08-01

In recent years significant progress have been made in the development of nonlinear data processing techniques. Such techniques are widely used in digital data filtering and image enhancement. Many of the most effective nonlinear filters based on order statistics. The widely used median filter is the best known order statistic filter. Generalized form of these filters could be presented based on Lloyd's statistics. Filters based on order statistics have excellent robustness properties in the presence of impulsive noise. In this paper, we present special approach for synthesis of order statistics filters using artificial neural networks. Optimal Lloyd's statistics are used for selecting of initial weights for the neural network. Adaptive properties of neural networks provide opportunities to optimize order statistics filters for data with asymmetric distribution function. Different examples demonstrate the properties and performance of presented approach.

Functional expansion representations of artificial neural networks

NASA Technical Reports Server (NTRS)

Gray, W. Steven

1992-01-01

In the past few years, significant interest has developed in using artificial neural networks to model and control nonlinear dynamical systems. While there exists many proposed schemes for accomplishing this and a wealth of supporting empirical results, most approaches to date tend to be ad hoc in nature and rely mainly on heuristic justifications. The purpose of this project was to further develop some analytical tools for representing nonlinear discrete-time input-output systems, which when applied to neural networks would give insight on architecture selection, pruning strategies, and learning algorithms. A long term goal is to determine in what sense, if any, a neural network can be used as a universal approximator for nonliner input-output maps with memory (i.e., realized by a dynamical system). This property is well known for the case of static or memoryless input-output maps. The general architecture under consideration in this project was a single-input, single-output recurrent feedforward network.

Generalised Transfer Functions of Neural Networks

NASA Astrophysics Data System (ADS)

Fung, C. F.; Billings, S. A.; Zhang, H.

1997-11-01

When artificial neural networks are used to model non-linear dynamical systems, the system structure which can be extremely useful for analysis and design, is buried within the network architecture. In this paper, explicit expressions for the frequency response or generalised transfer functions of both feedforward and recurrent neural networks are derived in terms of the network weights. The derivation of the algorithm is established on the basis of the Taylor series expansion of the activation functions used in a particular neural network. This leads to a representation which is equivalent to the non-linear recursive polynomial model and enables the derivation of the transfer functions to be based on the harmonic expansion method. By mapping the neural network into the frequency domain information about the structure of the underlying non-linear system can be recovered. Numerical examples are included to demonstrate the application of the new algorithm. These examples show that the frequency response functions appear to be highly sensitive to the network topology and training, and that the time domain properties fail to reveal deficiencies in the trained network structure.

Predictive Behavior of a Computational Foot/Ankle Model through Artificial Neural Networks.

PubMed

Chande, Ruchi D; Hargraves, Rosalyn Hobson; Ortiz-Robinson, Norma; Wayne, Jennifer S

2017-01-01

Computational models are useful tools to study the biomechanics of human joints. Their predictive performance is heavily dependent on bony anatomy and soft tissue properties. Imaging data provides anatomical requirements while approximate tissue properties are implemented from literature data, when available. We sought to improve the predictive capability of a computational foot/ankle model by optimizing its ligament stiffness inputs using feedforward and radial basis function neural networks. While the former demonstrated better performance than the latter per mean square error, both networks provided reasonable stiffness predictions for implementation into the computational model.

Modeling neural circuits in Parkinson's disease.

PubMed

Psiha, Maria; Vlamos, Panayiotis

2015-01-01

Parkinson's disease (PD) is caused by abnormal neural activity of the basal ganglia which are connected to the cerebral cortex in the brain surface through complex neural circuits. For a better understanding of the pathophysiological mechanisms of PD, it is important to identify the underlying PD neural circuits, and to pinpoint the precise nature of the crucial aberrations in these circuits. In this paper, the general architecture of a hybrid Multilayer Perceptron (MLP) network for modeling the neural circuits in PD is presented. The main idea of the proposed approach is to divide the parkinsonian neural circuitry system into three discrete subsystems: the external stimuli subsystem, the life-threatening events subsystem, and the basal ganglia subsystem. The proposed model, which includes the key roles of brain neural circuit in PD, is based on both feed-back and feed-forward neural networks. Specifically, a three-layer MLP neural network with feedback in the second layer was designed. The feedback in the second layer of this model simulates the dopamine modulatory effect of compacta on striatum.

On adaptive learning rate that guarantees convergence in feedforward networks.

PubMed

Behera, Laxmidhar; Kumar, Swagat; Patnaik, Awhan

2006-09-01

This paper investigates new learning algorithms (LF I and LF II) based on Lyapunov function for the training of feedforward neural networks. It is observed that such algorithms have interesting parallel with the popular backpropagation (BP) algorithm where the fixed learning rate is replaced by an adaptive learning rate computed using convergence theorem based on Lyapunov stability theory. LF II, a modified version of LF I, has been introduced with an aim to avoid local minima. This modification also helps in improving the convergence speed in some cases. Conditions for achieving global minimum for these kind of algorithms have been studied in detail. The performances of the proposed algorithms are compared with BP algorithm and extended Kalman filtering (EKF) on three bench-mark function approximation problems: XOR, 3-bit parity, and 8-3 encoder. The comparisons are made in terms of number of learning iterations and computational time required for convergence. It is found that the proposed algorithms (LF I and II) are much faster in convergence than other two algorithms to attain same accuracy. Finally, the comparison is made on a complex two-dimensional (2-D) Gabor function and effect of adaptive learning rate for faster convergence is verified. In a nutshell, the investigations made in this paper help us better understand the learning procedure of feedforward neural networks in terms of adaptive learning rate, convergence speed, and local minima.

Stimulus-specific adaptation in a recurrent network model of primary auditory cortex

PubMed Central

2017-01-01

Stimulus-specific adaptation (SSA) occurs when neurons decrease their responses to frequently-presented (standard) stimuli but not, or not as much, to other, rare (deviant) stimuli. SSA is present in all mammalian species in which it has been tested as well as in birds. SSA confers short-term memory to neuronal responses, and may lie upstream of the generation of mismatch negativity (MMN), an important human event-related potential. Previously published models of SSA mostly rely on synaptic depression of the feedforward, thalamocortical input. Here we study SSA in a recurrent neural network model of primary auditory cortex. When the recurrent, intracortical synapses display synaptic depression, the network generates population spikes (PSs). SSA occurs in this network when deviants elicit a PS but standards do not, and we demarcate the regions in parameter space that allow SSA. While SSA based on PSs does not require feedforward depression, we identify feedforward depression as a mechanism for expanding the range of parameters that support SSA. We provide predictions for experiments that could help differentiate between SSA due to synaptic depression of feedforward connections and SSA due to synaptic depression of recurrent connections. Similar to experimental data, the magnitude of SSA in the model depends on the frequency difference between deviant and standard, probability of the deviant, inter-stimulus interval and input amplitude. In contrast to models based on feedforward depression, our model shows true deviance sensitivity as found in experiments. PMID:28288158

High pressure air compressor valve fault diagnosis using feedforward neural networks

NASA Astrophysics Data System (ADS)

James Li, C.; Yu, Xueli

1995-09-01

Feedforward neural networks (FNNs) are developed and implemented to classify a four-stage high pressure air compressor into one of the following conditions: baseline, suction or exhaust valve faults. These FNNs are used for the compressor's automatic condition monitoring and fault diagnosis. Measurements of 39 variables are obtained under different baseline conditions and third-stage suction and exhaust valve faults. These variables include pressures and temperatures at all stages, voltage between phase aand phase b, voltage between phase band phase c, total three-phase real power, cooling water flow rate, etc. To reduce the number of variables, the amount of their discriminatory information is quantified by scattering matrices to identify statistical significant ones. Measurements of the selected variables are then used by a fully automatic structural and weight learning algorithm to construct three-layer FNNs to classify the compressor's condition. This learning algorithm requires neither guesses of initial weight values nor number of neurons in the hidden layer of an FNN. It takes an incremental approach in which a hidden neuron is trained by exemplars and then augmented to the existing network. These exemplars are then made orthogonal to the newly identified hidden neuron. They are subsequently used for the training of the next hidden neuron. The betterment continues until a desired accuracy is reached. After the neural networks are established, novel measurements from various conditions that haven't been previously seen by the FNNs are then used to evaluate their ability in fault diagnosis. The trained neural networks provide very accurate diagnosis for suction and discharge valve defects.

Programming an Artificial Neural Network Tool for Spatial Interpolation in GIS - A Case Study for Indoor Radio Wave Propagation of WLAN.

PubMed

Sen, Alper; Gümüsay, M Umit; Kavas, Aktül; Bulucu, Umut

2008-09-25

Wireless communication networks offer subscribers the possibilities of free mobility and access to information anywhere at any time. Therefore, electromagnetic coverage calculations are important for wireless mobile communication systems, especially in Wireless Local Area Networks (WLANs). Before any propagation computation is performed, modeling of indoor radio wave propagation needs accurate geographical information in order to avoid the interruption of data transmissions. Geographic Information Systems (GIS) and spatial interpolation techniques are very efficient for performing indoor radio wave propagation modeling. This paper describes the spatial interpolation of electromagnetic field measurements using a feed-forward back-propagation neural network programmed as a tool in GIS. The accuracy of Artificial Neural Networks (ANN) and geostatistical Kriging were compared by adjusting procedures. The feedforward back-propagation ANN provides adequate accuracy for spatial interpolation, but the predictions of Kriging interpolation are more accurate than the selected ANN. The proposed GIS ensures indoor radio wave propagation model and electromagnetic coverage, the number, position and transmitter power of access points and electromagnetic radiation level. Pollution analysis in a given propagation environment was done and it was demonstrated that WLAN (2.4 GHz) electromagnetic coverage does not lead to any electromagnetic pollution due to the low power levels used. Example interpolated electromagnetic field values for WLAN system in a building of Yildiz Technical University, Turkey, were generated using the selected network architectures to illustrate the results with an ANN.

Programming an Artificial Neural Network Tool for Spatial Interpolation in GIS - A Case Study for Indoor Radio Wave Propagation of WLAN

PubMed Central

Şen, Alper; Gümüşay, M. Ümit; Kavas, Aktül; Bulucu, Umut

2008-01-01

Wireless communication networks offer subscribers the possibilities of free mobility and access to information anywhere at any time. Therefore, electromagnetic coverage calculations are important for wireless mobile communication systems, especially in Wireless Local Area Networks (WLANs). Before any propagation computation is performed, modeling of indoor radio wave propagation needs accurate geographical information in order to avoid the interruption of data transmissions. Geographic Information Systems (GIS) and spatial interpolation techniques are very efficient for performing indoor radio wave propagation modeling. This paper describes the spatial interpolation of electromagnetic field measurements using a feed-forward back-propagation neural network programmed as a tool in GIS. The accuracy of Artificial Neural Networks (ANN) and geostatistical Kriging were compared by adjusting procedures. The feedforward back-propagation ANN provides adequate accuracy for spatial interpolation, but the predictions of Kriging interpolation are more accurate than the selected ANN. The proposed GIS ensures indoor radio wave propagation model and electromagnetic coverage, the number, position and transmitter power of access points and electromagnetic radiation level. Pollution analysis in a given propagation environment was done and it was demonstrated that WLAN (2.4 GHz) electromagnetic coverage does not lead to any electromagnetic pollution due to the low power levels used. Example interpolated electromagnetic field values for WLAN system in a building of Yildiz Technical University, Turkey, were generated using the selected network architectures to illustrate the results with an ANN. PMID:27873854

Numerical Simulation and Artificial Neural Network Modeling for Predicting Welding-Induced Distortion in Butt-Welded 304L Stainless Steel Plates

NASA Astrophysics Data System (ADS)

Narayanareddy, V. V.; Chandrasekhar, N.; Vasudevan, M.; Muthukumaran, S.; Vasantharaja, P.

2016-02-01

In the present study, artificial neural network modeling has been employed for predicting welding-induced angular distortions in autogenous butt-welded 304L stainless steel plates. The input data for the neural network have been obtained from a series of three-dimensional finite element simulations of TIG welding for a wide range of plate dimensions. Thermo-elasto-plastic analysis was carried out for 304L stainless steel plates during autogenous TIG welding employing double ellipsoidal heat source. The simulated thermal cycles were validated by measuring thermal cycles using thermocouples at predetermined positions, and the simulated distortion values were validated by measuring distortion using vertical height gauge for three cases. There was a good agreement between the model predictions and the measured values. Then, a multilayer feed-forward back propagation neural network has been developed using the numerically simulated data. Artificial neural network model developed in the present study predicted the angular distortion accurately.

Permutation parity machines for neural cryptography.

PubMed

Reyes, Oscar Mauricio; Zimmermann, Karl-Heinz

2010-06-01

Recently, synchronization was proved for permutation parity machines, multilayer feed-forward neural networks proposed as a binary variant of the tree parity machines. This ability was already used in the case of tree parity machines to introduce a key-exchange protocol. In this paper, a protocol based on permutation parity machines is proposed and its performance against common attacks (simple, geometric, majority and genetic) is studied.

Permutation parity machines for neural cryptography

DOE Office of Scientific and Technical Information (OSTI.GOV)

Reyes, Oscar Mauricio; Escuela de Ingenieria Electrica, Electronica y Telecomunicaciones, Universidad Industrial de Santander, Bucaramanga; Zimmermann, Karl-Heinz

2010-06-15

Recently, synchronization was proved for permutation parity machines, multilayer feed-forward neural networks proposed as a binary variant of the tree parity machines. This ability was already used in the case of tree parity machines to introduce a key-exchange protocol. In this paper, a protocol based on permutation parity machines is proposed and its performance against common attacks (simple, geometric, majority and genetic) is studied.

A neural network with modular hierarchical learning

NASA Technical Reports Server (NTRS)

Baldi, Pierre F. (Inventor); Toomarian, Nikzad (Inventor)

1994-01-01

This invention provides a new hierarchical approach for supervised neural learning of time dependent trajectories. The modular hierarchical methodology leads to architectures which are more structured than fully interconnected networks. The networks utilize a general feedforward flow of information and sparse recurrent connections to achieve dynamic effects. The advantages include the sparsity of units and connections, the modular organization. A further advantage is that the learning is much more circumscribed learning than in fully interconnected systems. The present invention is embodied by a neural network including a plurality of neural modules each having a pre-established performance capability wherein each neural module has an output outputting present results of the performance capability and an input for changing the present results of the performance capabilitiy. For pattern recognition applications, the performance capability may be an oscillation capability producing a repeating wave pattern as the present results. In the preferred embodiment, each of the plurality of neural modules includes a pre-established capability portion and a performance adjustment portion connected to control the pre-established capability portion.

Feedforward-Feedback Hybrid Control for Magnetic Shape Memory Alloy Actuators Based on the Krasnosel'skii-Pokrovskii Model

PubMed Central

Zhou, Miaolei; Zhang, Qi; Wang, Jingyuan

2014-01-01

As a new type of smart material, magnetic shape memory alloy has the advantages of a fast response frequency and outstanding strain capability in the field of microdrive and microposition actuators. The hysteresis nonlinearity in magnetic shape memory alloy actuators, however, limits system performance and further application. Here we propose a feedforward-feedback hybrid control method to improve control precision and mitigate the effects of the hysteresis nonlinearity of magnetic shape memory alloy actuators. First, hysteresis nonlinearity compensation for the magnetic shape memory alloy actuator is implemented by establishing a feedforward controller which is an inverse hysteresis model based on Krasnosel'skii-Pokrovskii operator. Secondly, the paper employs the classical Proportion Integration Differentiation feedback control with feedforward control to comprise the hybrid control system, and for further enhancing the adaptive performance of the system and improving the control accuracy, the Radial Basis Function neural network self-tuning Proportion Integration Differentiation feedback control replaces the classical Proportion Integration Differentiation feedback control. Utilizing self-learning ability of the Radial Basis Function neural network obtains Jacobian information of magnetic shape memory alloy actuator for the on-line adjustment of parameters in Proportion Integration Differentiation controller. Finally, simulation results show that the hybrid control method proposed in this paper can greatly improve the control precision of magnetic shape memory alloy actuator and the maximum tracking error is reduced from 1.1% in the open-loop system to 0.43% in the hybrid control system. PMID:24828010

Feedforward-feedback hybrid control for magnetic shape memory alloy actuators based on the Krasnosel'skii-Pokrovskii model.

PubMed

Zhou, Miaolei; Zhang, Qi; Wang, Jingyuan

2014-01-01

As a new type of smart material, magnetic shape memory alloy has the advantages of a fast response frequency and outstanding strain capability in the field of microdrive and microposition actuators. The hysteresis nonlinearity in magnetic shape memory alloy actuators, however, limits system performance and further application. Here we propose a feedforward-feedback hybrid control method to improve control precision and mitigate the effects of the hysteresis nonlinearity of magnetic shape memory alloy actuators. First, hysteresis nonlinearity compensation for the magnetic shape memory alloy actuator is implemented by establishing a feedforward controller which is an inverse hysteresis model based on Krasnosel'skii-Pokrovskii operator. Secondly, the paper employs the classical Proportion Integration Differentiation feedback control with feedforward control to comprise the hybrid control system, and for further enhancing the adaptive performance of the system and improving the control accuracy, the Radial Basis Function neural network self-tuning Proportion Integration Differentiation feedback control replaces the classical Proportion Integration Differentiation feedback control. Utilizing self-learning ability of the Radial Basis Function neural network obtains Jacobian information of magnetic shape memory alloy actuator for the on-line adjustment of parameters in Proportion Integration Differentiation controller. Finally, simulation results show that the hybrid control method proposed in this paper can greatly improve the control precision of magnetic shape memory alloy actuator and the maximum tracking error is reduced from 1.1% in the open-loop system to 0.43% in the hybrid control system.

Prediction of Flow Stress in Cadmium Using Constitutive Equation and Artificial Neural Network Approach

NASA Astrophysics Data System (ADS)

Sarkar, A.; Chakravartty, J. K.

2013-10-01

A model is developed to predict the constitutive flow behavior of cadmium during compression test using artificial neural network (ANN). The inputs of the neural network are strain, strain rate, and temperature, whereas flow stress is the output. Experimental data obtained from compression tests in the temperature range -30 to 70 °C, strain range 0.1 to 0.6, and strain rate range 10-3 to 1 s-1 are employed to develop the model. A three-layer feed-forward ANN is trained with Levenberg-Marquardt training algorithm. It has been shown that the developed ANN model can efficiently and accurately predict the deformation behavior of cadmium. This trained network could predict the flow stress better than a constitutive equation of the type.

Optimal mapping of neural-network learning on message-passing multicomputers

NASA Technical Reports Server (NTRS)

Chu, Lon-Chan; Wah, Benjamin W.

1992-01-01

A minimization of learning-algorithm completion time is sought in the present optimal-mapping study of the learning process in multilayer feed-forward artificial neural networks (ANNs) for message-passing multicomputers. A novel approximation algorithm for mappings of this kind is derived from observations of the dominance of a parallel ANN algorithm over its communication time. Attention is given to both static and dynamic mapping schemes for systems with static and dynamic background workloads, as well as to experimental results obtained for simulated mappings on multicomputers with dynamic background workloads.

Feedforward, high density, programmable read only neural network based memory system

NASA Technical Reports Server (NTRS)

Daud, Taher; Moopenn, Alex; Lamb, James; Thakoor, Anil; Khanna, Satish

1988-01-01

Neural network-inspired, nonvolatile, programmable associative memory using thin-film technology is demonstrated. The details of the architecture, which uses programmable resistive connection matrices in synaptic arrays and current summing and thresholding amplifiers as neurons, are described. Several synapse configurations for a high-density array of a binary connection matrix are also described. Test circuits are evaluated for operational feasibility and to demonstrate the speed of the read operation. The results are discussed to highlight the potential for a read data rate exceeding 10 megabits/sec.

Supervised learning of probability distributions by neural networks

NASA Technical Reports Server (NTRS)

Baum, Eric B.; Wilczek, Frank

1988-01-01

Supervised learning algorithms for feedforward neural networks are investigated analytically. The back-propagation algorithm described by Werbos (1974), Parker (1985), and Rumelhart et al. (1986) is generalized by redefining the values of the input and output neurons as probabilities. The synaptic weights are then varied to follow gradients in the logarithm of likelihood rather than in the error. This modification is shown to provide a more rigorous theoretical basis for the algorithm and to permit more accurate predictions. A typical application involving a medical-diagnosis expert system is discussed.

Learning the Relationship between the Primary Structure of HIV Envelope Glycoproteins and Neutralization Activity of Particular Antibodies by Using Artificial Neural Networks

PubMed Central

Buiu, Cătălin; Putz, Mihai V.; Avram, Speranta

2016-01-01

The dependency between the primary structure of HIV envelope glycoproteins (ENV) and the neutralization data for given antibodies is very complicated and depends on a large number of factors, such as the binding affinity of a given antibody for a given ENV protein, and the intrinsic infection kinetics of the viral strain. This paper presents a first approach to learning these dependencies using an artificial feedforward neural network which is trained to learn from experimental data. The results presented here demonstrate that the trained neural network is able to generalize on new viral strains and to predict reliable values of neutralizing activities of given antibodies against HIV-1. PMID:27727189

Using wavelet-feedforward neural networks to improve air pollution forecasting in urban environments.

PubMed

Dunea, Daniel; Pohoata, Alin; Iordache, Stefania

2015-07-01

The paper presents the screening of various feedforward neural networks (FANN) and wavelet-feedforward neural networks (WFANN) applied to time series of ground-level ozone (O3), nitrogen dioxide (NO2), and particulate matter (PM10 and PM2.5 fractions) recorded at four monitoring stations located in various urban areas of Romania, to identify common configurations with optimal generalization performance. Two distinct model runs were performed as follows: data processing using hourly-recorded time series of airborne pollutants during cold months (O3, NO2, and PM10), when residential heating increases the local emissions, and data processing using 24-h daily averaged concentrations (PM2.5) recorded between 2009 and 2012. Dataset variability was assessed using statistical analysis. Time series were passed through various FANNs. Each time series was decomposed in four time-scale components using three-level wavelets, which have been passed also through FANN, and recomposed into a single time series. The agreement between observed and modelled output was evaluated based on the statistical significance (r coefficient and correlation between errors and data). Daubechies db3 wavelet-Rprop FANN (6-4-1) utilization gave positive results for O3 time series optimizing the exclusive use of the FANN for hourly-recorded time series. NO2 was difficult to model due to time series specificity, but wavelet integration improved FANN performances. Daubechies db3 wavelet did not improve the FANN outputs for PM10 time series. Both models (FANN/WFANN) overestimated PM2.5 forecasted values in the last quarter of time series. A potential improvement of the forecasted values could be the integration of a smoothing algorithm to adjust the PM2.5 model outputs.

Feasibility study of robotic neural controllers

NASA Technical Reports Server (NTRS)

Magana, Mario E.

1990-01-01

The results are given of a feasibility study performed to establish if an artificial neural controller could be used to achieve joint space trajectory tracking of a two-link robot manipulator. The study is based on the results obtained by Hecht-Nielsen, who claims that a functional map can be implemented to a desired degree of accuracy with a three layer feedforward artificial neural network. Central to this study is the assumption that the robot model as well as its parameters values are known.

An application of artificial neural networks to experimental data approximation

NASA Technical Reports Server (NTRS)

Meade, Andrew J., Jr.

1993-01-01

As an initial step in the evaluation of networks, a feedforward architecture is trained to approximate experimental data by the backpropagation algorithm. Several drawbacks were detected and an alternative learning algorithm was then developed to partially address the drawbacks. This noniterative algorithm has a number of advantages over the backpropagation method and is easily implemented on existing hardware.

Using soft computing techniques to predict corrected air permeability using Thomeer parameters, air porosity and grain density

NASA Astrophysics Data System (ADS)

Nooruddin, Hasan A.; Anifowose, Fatai; Abdulraheem, Abdulazeez

2014-03-01

Soft computing techniques are recently becoming very popular in the oil industry. A number of computational intelligence-based predictive methods have been widely applied in the industry with high prediction capabilities. Some of the popular methods include feed-forward neural networks, radial basis function network, generalized regression neural network, functional networks, support vector regression and adaptive network fuzzy inference system. A comparative study among most popular soft computing techniques is presented using a large dataset published in literature describing multimodal pore systems in the Arab D formation. The inputs to the models are air porosity, grain density, and Thomeer parameters obtained using mercury injection capillary pressure profiles. Corrected air permeability is the target variable. Applying developed permeability models in recent reservoir characterization workflow ensures consistency between micro and macro scale information represented mainly by Thomeer parameters and absolute permeability. The dataset was divided into two parts with 80% of data used for training and 20% for testing. The target permeability variable was transformed to the logarithmic scale as a pre-processing step and to show better correlations with the input variables. Statistical and graphical analysis of the results including permeability cross-plots and detailed error measures were created. In general, the comparative study showed very close results among the developed models. The feed-forward neural network permeability model showed the lowest average relative error, average absolute relative error, standard deviations of error and root means squares making it the best model for such problems. Adaptive network fuzzy inference system also showed very good results.

Feed-Forward Propagation of Temporal and Rate Information between Cortical Populations during Coherent Activation in Engineered In Vitro Networks.

PubMed

DeMarse, Thomas B; Pan, Liangbin; Alagapan, Sankaraleengam; Brewer, Gregory J; Wheeler, Bruce C

2016-01-01

Transient propagation of information across neuronal assembles is thought to underlie many cognitive processes. However, the nature of the neural code that is embedded within these transmissions remains uncertain. Much of our understanding of how information is transmitted among these assemblies has been derived from computational models. While these models have been instrumental in understanding these processes they often make simplifying assumptions about the biophysical properties of neurons that may influence the nature and properties expressed. To address this issue we created an in vitro analog of a feed-forward network composed of two small populations (also referred to as assemblies or layers) of living dissociated rat cortical neurons. The populations were separated by, and communicated through, a microelectromechanical systems (MEMS) device containing a strip of microscale tunnels. Delayed culturing of one population in the first layer followed by the second a few days later induced the unidirectional growth of axons through the microtunnels resulting in a primarily feed-forward communication between these two small neural populations. In this study we systematically manipulated the number of tunnels that connected each layer and hence, the number of axons providing communication between those populations. We then assess the effect of reducing the number of tunnels has upon the properties of between-layer communication capacity and fidelity of neural transmission among spike trains transmitted across and within layers. We show evidence based on Victor-Purpura's and van Rossum's spike train similarity metrics supporting the presence of both rate and temporal information embedded within these transmissions whose fidelity increased during communication both between and within layers when the number of tunnels are increased. We also provide evidence reinforcing the role of synchronized activity upon transmission fidelity during the spontaneous synchronized network burst events that propagated between layers and highlight the potential applications of these MEMs devices as a tool for further investigation of structure and functional dynamics among neural populations.

Feed-Forward Propagation of Temporal and Rate Information between Cortical Populations during Coherent Activation in Engineered In Vitro Networks

PubMed Central

DeMarse, Thomas B.; Pan, Liangbin; Alagapan, Sankaraleengam; Brewer, Gregory J.; Wheeler, Bruce C.

2016-01-01

Transient propagation of information across neuronal assembles is thought to underlie many cognitive processes. However, the nature of the neural code that is embedded within these transmissions remains uncertain. Much of our understanding of how information is transmitted among these assemblies has been derived from computational models. While these models have been instrumental in understanding these processes they often make simplifying assumptions about the biophysical properties of neurons that may influence the nature and properties expressed. To address this issue we created an in vitro analog of a feed-forward network composed of two small populations (also referred to as assemblies or layers) of living dissociated rat cortical neurons. The populations were separated by, and communicated through, a microelectromechanical systems (MEMS) device containing a strip of microscale tunnels. Delayed culturing of one population in the first layer followed by the second a few days later induced the unidirectional growth of axons through the microtunnels resulting in a primarily feed-forward communication between these two small neural populations. In this study we systematically manipulated the number of tunnels that connected each layer and hence, the number of axons providing communication between those populations. We then assess the effect of reducing the number of tunnels has upon the properties of between-layer communication capacity and fidelity of neural transmission among spike trains transmitted across and within layers. We show evidence based on Victor-Purpura’s and van Rossum’s spike train similarity metrics supporting the presence of both rate and temporal information embedded within these transmissions whose fidelity increased during communication both between and within layers when the number of tunnels are increased. We also provide evidence reinforcing the role of synchronized activity upon transmission fidelity during the spontaneous synchronized network burst events that propagated between layers and highlight the potential applications of these MEMs devices as a tool for further investigation of structure and functional dynamics among neural populations. PMID:27147977

Intelligent processing of acoustic emission signals

NASA Astrophysics Data System (ADS)

Sachse, Wolfgang; Grabec, Igor

1992-07-01

Recent developments in applying neural-like signal-processing procedures for analyzing acoustic emission signals are summarized. These procedures employ a set of learning signals to develop a memory that can subsequently be utilized to process other signals to recover information about an unknown source. A majority of the current applications to process ultrasonic waveforms are based on multilayered, feed-forward neural networks, trained with some type of back-error propagation rule.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Derrida, B.; Meir, R.

We consider the evolution of configurations in a layered feed-forward neural network. Exact expressions for the evolution of the distance between two configurations are obtained in the thermodynamic limit. Our results show that the distance between two arbitrarily close configurations always increases, implying chaotic behavior, even in the phase of good retrieval.

Development of Sorting System for Fishes by Feed-forward Neural Networks Using Rotation Invariant Features

NASA Astrophysics Data System (ADS)

Shiraishi, Yuhki; Takeda, Fumiaki

In this research, we have developed a sorting system for fishes, which is comprised of a conveyance part, a capturing image part, and a sorting part. In the conveyance part, we have developed an independent conveyance system in order to separate one fish from an intertwined group of fishes. After the image of the separated fish is captured in the capturing part, a rotation invariant feature is extracted using two-dimensional fast Fourier transform, which is the mean value of the power spectrum with the same distance from the origin in the spectrum field. After that, the fishes are classified by three-layered feed-forward neural networks. The experimental results show that the developed system classifies three kinds of fishes captured in various angles with the classification ratio of 98.95% for 1044 captured images of five fishes. The other experimental results show the classification ratio of 90.7% for 300 fishes by 10-fold cross validation method.

An H(∞) control approach to robust learning of feedforward neural networks.

PubMed

Jing, Xingjian

2011-09-01

A novel H(∞) robust control approach is proposed in this study to deal with the learning problems of feedforward neural networks (FNNs). The analysis and design of a desired weight update law for the FNN is transformed into a robust controller design problem for a discrete dynamic system in terms of the estimation error. The drawbacks of some existing learning algorithms can therefore be revealed, especially for the case that the output data is fast changing with respect to the input or the output data is corrupted by noise. Based on this approach, the optimal learning parameters can be found by utilizing the linear matrix inequality (LMI) optimization techniques to achieve a predefined H(∞) "noise" attenuation level. Several existing BP-type algorithms are shown to be special cases of the new H(∞)-learning algorithm. Theoretical analysis and several examples are provided to show the advantages of the new method. Copyright © 2011 Elsevier Ltd. All rights reserved.

On the stability, storage capacity, and design of nonlinear continuous neural networks

NASA Technical Reports Server (NTRS)

Guez, Allon; Protopopsecu, Vladimir; Barhen, Jacob

1988-01-01

The stability, capacity, and design of a nonlinear continuous neural network are analyzed. Sufficient conditions for existence and asymptotic stability of the network's equilibria are reduced to a set of piecewise-linear inequality relations that can be solved by a feedforward binary network, or by methods such as Fourier elimination. The stability and capacity of the network is characterized by the post synaptic firing rate function. An N-neuron network with sigmoidal firing function is shown to have up to 3N equilibrium points. This offers a higher capacity than the (0.1-0.2)N obtained in the binary Hopfield network. Moreover, it is shown that by a proper selection of the postsynaptic firing rate function, one can significantly extend the capacity storage of the network.

Character Recognition Using Genetically Trained Neural Networks

DOE Office of Scientific and Technical Information (OSTI.GOV)

Diniz, C.; Stantz, K.M.; Trahan, M.W.

1998-10-01

Computationally intelligent recognition of characters and symbols addresses a wide range of applications including foreign language translation and chemical formula identification. The combination of intelligent learning and optimization algorithms with layered neural structures offers powerful techniques for character recognition. These techniques were originally developed by Sandia National Laboratories for pattern and spectral analysis; however, their ability to optimize vast amounts of data make them ideal for character recognition. An adaptation of the Neural Network Designer soflsvare allows the user to create a neural network (NN_) trained by a genetic algorithm (GA) that correctly identifies multiple distinct characters. The initial successfidmore » recognition of standard capital letters can be expanded to include chemical and mathematical symbols and alphabets of foreign languages, especially Arabic and Chinese. The FIN model constructed for this project uses a three layer feed-forward architecture. To facilitate the input of characters and symbols, a graphic user interface (GUI) has been developed to convert the traditional representation of each character or symbol to a bitmap. The 8 x 8 bitmap representations used for these tests are mapped onto the input nodes of the feed-forward neural network (FFNN) in a one-to-one correspondence. The input nodes feed forward into a hidden layer, and the hidden layer feeds into five output nodes correlated to possible character outcomes. During the training period the GA optimizes the weights of the NN until it can successfully recognize distinct characters. Systematic deviations from the base design test the network's range of applicability. Increasing capacity, the number of letters to be recognized, requires a nonlinear increase in the number of hidden layer neurodes. Optimal character recognition performance necessitates a minimum threshold for the number of cases when genetically training the net. And, the amount of noise significantly degrades character recognition efficiency, some of which can be overcome by adding noise during training and optimizing the form of the network's activation fimction.« less

Particle identification with neural networks using a rotational invariant moment representation

NASA Astrophysics Data System (ADS)

Sinkus, Ralph; Voss, Thomas

1997-02-01

A feed-forward neural network is used to identify electromagnetic particles based upon their showering properties within a segmented calorimeter. A preprocessing procedure is applied to the spatial energy distribution of the particle shower in order to account for the varying geometry of the calorimeter. The novel feature is the expansion of the energy distribution in terms of moments of the so-called Zernike functions which are invariant under rotation. The distributions of moments exhibit very different scales, thus the multidimensional input distribution for the neural network is transformed via a principal component analysis and rescaled by its respective variances to ensure input values of the order of one. This increases the sensitivity of the network and thus results in better performance in identifying and separating electromagnetic from hadronic particles, especially at low energies.

Toward a More Robust Pruning Procedure for MLP Networks

NASA Technical Reports Server (NTRS)

Stepniewski, Slawomir W.; Jorgensen, Charles C.

1998-01-01

Choosing a proper neural network architecture is a problem of great practical importance. Smaller models mean not only simpler designs but also lower variance for parameter estimation and network prediction. The widespread utilization of neural networks in modeling highlights an issue in human factors. The procedure of building neural models should find an appropriate level of model complexity in a more or less automatic fashion to make it less prone to human subjectivity. In this paper we present a Singular Value Decomposition based node elimination technique and enhanced implementation of the Optimal Brain Surgeon algorithm. Combining both methods creates a powerful pruning engine that can be used for tuning feedforward connectionist models. The performance of the proposed method is demonstrated by adjusting the structure of a multi-input multi-output model used to calibrate a six-component wind tunnel strain gage.

Combined feedforward and feedback control of a redundant, nonlinear, dynamic musculoskeletal system.

PubMed

Blana, Dimitra; Kirsch, Robert F; Chadwick, Edward K

2009-05-01

A functional electrical stimulation controller is presented that uses a combination of feedforward and feedback for arm control in high-level injury. The feedforward controller generates the muscle activations nominally required for desired movements, and the feedback controller corrects for errors caused by muscle fatigue and external disturbances. The feedforward controller is an artificial neural network (ANN) which approximates the inverse dynamics of the arm. The feedback loop includes a PID controller in series with a second ANN representing the nonlinear properties and biomechanical interactions of muscles and joints. The controller was designed and tested using a two-joint musculoskeletal model of the arm that includes four mono-articular and two bi-articular muscles. Its performance during goal-oriented movements of varying amplitudes and durations showed a tracking error of less than 4 degrees in ideal conditions, and less than 10 degrees even in the case of considerable fatigue and external disturbances.

Training trajectories by continuous recurrent multilayer networks.

PubMed

Leistritz, L; Galicki, M; Witte, H; Kochs, E

2002-01-01

This paper addresses the problem of training trajectories by means of continuous recurrent neural networks whose feedforward parts are multilayer perceptrons. Such networks can approximate a general nonlinear dynamic system with arbitrary accuracy. The learning process is transformed into an optimal control framework where the weights are the controls to be determined. A training algorithm based upon a variational formulation of Pontryagin's maximum principle is proposed for such networks. Computer examples demonstrating the efficiency of the given approach are also presented.

Predicting Physical Time Series Using Dynamic Ridge Polynomial Neural Networks

PubMed Central

Al-Jumeily, Dhiya; Ghazali, Rozaida; Hussain, Abir

2014-01-01

Forecasting naturally occurring phenomena is a common problem in many domains of science, and this has been addressed and investigated by many scientists. The importance of time series prediction stems from the fact that it has wide range of applications, including control systems, engineering processes, environmental systems and economics. From the knowledge of some aspects of the previous behaviour of the system, the aim of the prediction process is to determine or predict its future behaviour. In this paper, we consider a novel application of a higher order polynomial neural network architecture called Dynamic Ridge Polynomial Neural Network that combines the properties of higher order and recurrent neural networks for the prediction of physical time series. In this study, four types of signals have been used, which are; The Lorenz attractor, mean value of the AE index, sunspot number, and heat wave temperature. The simulation results showed good improvements in terms of the signal to noise ratio in comparison to a number of higher order and feedforward neural networks in comparison to the benchmarked techniques. PMID:25157950

Predicting physical time series using dynamic ridge polynomial neural networks.

PubMed

Al-Jumeily, Dhiya; Ghazali, Rozaida; Hussain, Abir

2014-01-01

Forecasting naturally occurring phenomena is a common problem in many domains of science, and this has been addressed and investigated by many scientists. The importance of time series prediction stems from the fact that it has wide range of applications, including control systems, engineering processes, environmental systems and economics. From the knowledge of some aspects of the previous behaviour of the system, the aim of the prediction process is to determine or predict its future behaviour. In this paper, we consider a novel application of a higher order polynomial neural network architecture called Dynamic Ridge Polynomial Neural Network that combines the properties of higher order and recurrent neural networks for the prediction of physical time series. In this study, four types of signals have been used, which are; The Lorenz attractor, mean value of the AE index, sunspot number, and heat wave temperature. The simulation results showed good improvements in terms of the signal to noise ratio in comparison to a number of higher order and feedforward neural networks in comparison to the benchmarked techniques.

Development of a computational model on the neural activity patterns of a visual working memory in a hierarchical feedforward Network

NASA Astrophysics Data System (ADS)

An, Soyoung; Choi, Woochul; Paik, Se-Bum

2015-11-01

Understanding the mechanism of information processing in the human brain remains a unique challenge because the nonlinear interactions between the neurons in the network are extremely complex and because controlling every relevant parameter during an experiment is difficult. Therefore, a simulation using simplified computational models may be an effective approach. In the present study, we developed a general model of neural networks that can simulate nonlinear activity patterns in the hierarchical structure of a neural network system. To test our model, we first examined whether our simulation could match the previously-observed nonlinear features of neural activity patterns. Next, we performed a psychophysics experiment for a simple visual working memory task to evaluate whether the model could predict the performance of human subjects. Our studies show that the model is capable of reproducing the relationship between memory load and performance and may contribute, in part, to our understanding of how the structure of neural circuits can determine the nonlinear neural activity patterns in the human brain.

Minimal perceptrons for memorizing complex patterns

NASA Astrophysics Data System (ADS)

Pastor, Marissa; Song, Juyong; Hoang, Danh-Tai; Jo, Junghyo

2016-11-01

Feedforward neural networks have been investigated to understand learning and memory, as well as applied to numerous practical problems in pattern classification. It is a rule of thumb that more complex tasks require larger networks. However, the design of optimal network architectures for specific tasks is still an unsolved fundamental problem. In this study, we consider three-layered neural networks for memorizing binary patterns. We developed a new complexity measure of binary patterns, and estimated the minimal network size for memorizing them as a function of their complexity. We formulated the minimal network size for regular, random, and complex patterns. In particular, the minimal size for complex patterns, which are neither ordered nor disordered, was predicted by measuring their Hamming distances from known ordered patterns. Our predictions agree with simulations based on the back-propagation algorithm.

Neuromorphic Learning From Noisy Data

NASA Technical Reports Server (NTRS)

Merrill, Walter C.; Troudet, Terry

1993-01-01

Two reports present numerical study of performance of feedforward neural network trained by back-propagation algorithm in learning continuous-valued mappings from data corrupted by noise. Two types of noise considered: plant noise which affects dynamics of controlled process and data-processing noise, which occurs during analog processing and digital sampling of signals. Study performed with view toward use of neural networks as neurocontrollers to substitute for, or enhance, performances of human experts in controlling mechanical devices in presence of sensor and actuator noise and to enhance performances of more-conventional digital feedback electronic process controllers in noisy environments.

Machine Learning Technique to Find Quantum Many-Body Ground States of Bosons on a Lattice

NASA Astrophysics Data System (ADS)

Saito, Hiroki; Kato, Masaya

2018-01-01

We have developed a variational method to obtain many-body ground states of the Bose-Hubbard model using feedforward artificial neural networks. A fully connected network with a single hidden layer works better than a fully connected network with multiple hidden layers, and a multilayer convolutional network is more efficient than a fully connected network. AdaGrad and Adam are optimization methods that work well. Moreover, we show that many-body ground states with different numbers of particles can be generated by a single network.

Efficient self-organizing multilayer neural network for nonlinear system modeling.

PubMed

Han, Hong-Gui; Wang, Li-Dan; Qiao, Jun-Fei

2013-07-01

It has been shown extensively that the dynamic behaviors of a neural system are strongly influenced by the network architecture and learning process. To establish an artificial neural network (ANN) with self-organizing architecture and suitable learning algorithm for nonlinear system modeling, an automatic axon-neural network (AANN) is investigated in the following respects. First, the network architecture is constructed automatically to change both the number of hidden neurons and topologies of the neural network during the training process. The approach introduced in adaptive connecting-and-pruning algorithm (ACP) is a type of mixed mode operation, which is equivalent to pruning or adding the connecting of the neurons, as well as inserting some required neurons directly. Secondly, the weights are adjusted, using a feedforward computation (FC) to obtain the information for the gradient during learning computation. Unlike most of the previous studies, AANN is able to self-organize the architecture and weights, and to improve the network performances. Also, the proposed AANN has been tested on a number of benchmark problems, ranging from nonlinear function approximating to nonlinear systems modeling. The experimental results show that AANN can have better performances than that of some existing neural networks. Crown Copyright © 2013. Published by Elsevier Ltd. All rights reserved.

Neural substrates of visuomotor learning based on improved feedback control and prediction

PubMed Central

Grafton, Scott T.; Schmitt, Paul; Horn, John Van; Diedrichsen, Jörn

2008-01-01

Motor skills emerge from learning feedforward commands as well as improvements in feedback control. These two components of learning were investigated in a compensatory visuomotor tracking task on a trial-by-trial basis. Between trial learning was characterized with a state-space model to provide smoothed estimates of feedforward and feedback learning, separable from random fluctuations in motor performance and error. The resultant parameters were correlated with brain activity using magnetic resonance imaging. Learning related to the generation of a feedforward command correlated with activity in dorsal premotor cortex, inferior parietal lobule, supplementary motor area and cingulate motor area, supporting a role of these areas in retrieving and executing a predictive motor command. Modulation of feedback control was associated with activity in bilateral posterior superior parietal lobule as well as right ventral premotor cortex. Performance error correlated with activity in a widespread cortical and subcortical network including bilateral parietal, premotor and rostral anterior cingulate cortex as well as the cerebellar cortex. Finally, trial-by-trial changes of kinematics, as measured by mean absolute hand acceleration, correlated with activity in motor cortex and anterior cerebellum. The results demonstrate that incremental, learning dependent changes can be modeled on a trial-by-trial basis and neural substrates for feedforward control of novel motor programs are localized to secondary motor areas. PMID:18032069

Distributed multisensory integration in a recurrent network model through supervised learning

NASA Astrophysics Data System (ADS)

Wang, He; Wong, K. Y. Michael

Sensory integration between different modalities has been extensively studied. It is suggested that the brain integrates signals from different modalities in a Bayesian optimal way. However, how the Bayesian rule is implemented in a neural network remains under debate. In this work we propose a biologically plausible recurrent network model, which can perform Bayesian multisensory integration after trained by supervised learning. Our model is composed of two modules, each for one modality. We assume that each module is a recurrent network, whose activity represents the posterior distribution of each stimulus. The feedforward input on each module is the likelihood of each modality. Two modules are integrated through cross-links, which are feedforward connections from the other modality, and reciprocal connections, which are recurrent connections between different modules. By stochastic gradient descent, we successfully trained the feedforward and recurrent coupling matrices simultaneously, both of which resembles the Mexican-hat. We also find that there are more than one set of coupling matrices that can approximate the Bayesian theorem well. Specifically, reciprocal connections and cross-links will compensate each other if one of them is removed. Even though trained with two inputs, the network's performance with only one input is in good accordance with what is predicted by the Bayesian theorem.

Maximum entropy methods for extracting the learned features of deep neural networks.

PubMed

Finnegan, Alex; Song, Jun S

2017-10-01

New architectures of multilayer artificial neural networks and new methods for training them are rapidly revolutionizing the application of machine learning in diverse fields, including business, social science, physical sciences, and biology. Interpreting deep neural networks, however, currently remains elusive, and a critical challenge lies in understanding which meaningful features a network is actually learning. We present a general method for interpreting deep neural networks and extracting network-learned features from input data. We describe our algorithm in the context of biological sequence analysis. Our approach, based on ideas from statistical physics, samples from the maximum entropy distribution over possible sequences, anchored at an input sequence and subject to constraints implied by the empirical function learned by a network. Using our framework, we demonstrate that local transcription factor binding motifs can be identified from a network trained on ChIP-seq data and that nucleosome positioning signals are indeed learned by a network trained on chemical cleavage nucleosome maps. Imposing a further constraint on the maximum entropy distribution also allows us to probe whether a network is learning global sequence features, such as the high GC content in nucleosome-rich regions. This work thus provides valuable mathematical tools for interpreting and extracting learned features from feed-forward neural networks.

Dispersion compensation of fiber optic communication system with direct detection using artificial neural networks (ANNs)

NASA Astrophysics Data System (ADS)

Maghrabi, Mahmoud M. T.; Kumar, Shiva; Bakr, Mohamed H.

2018-02-01

This work introduces a powerful digital nonlinear feed-forward equalizer (NFFE), exploiting multilayer artificial neural network (ANN). It mitigates impairments of optical communication systems arising due to the nonlinearity introduced by direct photo-detection. In a direct detection system, the detection process is nonlinear due to the fact that the photo-current is proportional to the absolute square of the electric field intensity. The proposed equalizer provides the most efficient computational cost with high equalization performance. Its performance is comparable to the benchmark compensation performance achieved by maximum-likelihood sequence estimator. The equalizer trains an ANN to act as a nonlinear filter whose impulse response removes the intersymbol interference (ISI) distortions of the optical channel. Owing to the proposed extensive training of the equalizer, it achieves the ultimate performance limit of any feed-forward equalizer (FFE). The performance and efficiency of the equalizer is investigated by applying it to various practical short-reach fiber optic communication system scenarios. These scenarios are extracted from practical metro/media access networks and data center applications. The obtained results show that the ANN-NFFE compensates for the received BER degradation and significantly increases the tolerance to the chromatic dispersion distortion.

Adaptive Neuron Model: An architecture for the rapid learning of nonlinear topological transformations

NASA Technical Reports Server (NTRS)

Tawel, Raoul (Inventor)

1994-01-01

A method for the rapid learning of nonlinear mappings and topological transformations using a dynamically reconfigurable artificial neural network is presented. This fully-recurrent Adaptive Neuron Model (ANM) network was applied to the highly degenerate inverse kinematics problem in robotics, and its performance evaluation is bench-marked. Once trained, the resulting neuromorphic architecture was implemented in custom analog neural network hardware and the parameters capturing the functional transformation downloaded onto the system. This neuroprocessor, capable of 10(exp 9) ops/sec, was interfaced directly to a three degree of freedom Heathkit robotic manipulator. Calculation of the hardware feed-forward pass for this mapping was benchmarked at approximately 10 microsec.

Statistical inference, the bootstrap, and neural-network modeling with application to foreign exchange rates.

PubMed

White, H; Racine, J

2001-01-01

We propose tests for individual and joint irrelevance of network inputs. Such tests can be used to determine whether an input or group of inputs "belong" in a particular model, thus permitting valid statistical inference based on estimated feedforward neural-network models. The approaches employ well-known statistical resampling techniques. We conduct a small Monte Carlo experiment showing that our tests have reasonable level and power behavior, and we apply our methods to examine whether there are predictable regularities in foreign exchange rates. We find that exchange rates do appear to contain information that is exploitable for enhanced point prediction, but the nature of the predictive relations evolves through time.

A novel and generalized approach in the inversion of geoelectrical resistivity data using Artificial Neural Networks (ANN)

NASA Astrophysics Data System (ADS)

Raj, A. Stanley; Srinivas, Y.; Oliver, D. Hudson; Muthuraj, D.

2014-03-01

The non-linear apparent resistivity problem in the subsurface study of the earth takes into account the model parameters in terms of resistivity and thickness of individual subsurface layers using the trained synthetic data by means of Artificial Neural Networks (ANN). Here we used a single layer feed-forward neural network with fast back propagation learning algorithm. So on proper training of back propagation networks it tends to give the resistivity and thickness of the subsurface layer model of the field resistivity data with reference to the synthetic data trained in the appropriate network. During training, the weights and biases of the network are iteratively adjusted to make network performance function level more efficient. On adequate training, errors are minimized and the best result is obtained using the artificial neural networks. The network is trained with more number of VES data and this trained network is demonstrated by the field data. The accuracy of inversion depends upon the number of data trained. In this novel and specially designed algorithm, the interpretation of the vertical electrical sounding has been done successfully with the more accurate layer model.

Artificial neural networks for modeling time series of beach litter in the southern North Sea.

PubMed

Schulz, Marcus; Matthies, Michael

2014-07-01

In European marine waters, existing monitoring programs of beach litter need to be improved concerning litter items used as indicators of pollution levels, efficiency, and effectiveness. In order to ease and focus future monitoring of beach litter on few important litter items, feed-forward neural networks consisting of three layers were developed to relate single litter items to general categories of marine litter. The neural networks developed were applied to seven beaches in the southern North Sea and modeled time series of five general categories of marine litter, such as litter from fishing, shipping, and tourism. Results of regression analyses show that general categories were predicted significantly moderately to well. Measured and modeled data were in the same order of magnitude, and minima and maxima overlapped well. Neural networks were found to be eligible tools to deliver reliable predictions of marine litter with low computational effort and little input of information. Copyright © 2014 Elsevier Ltd. All rights reserved.

A comparison between HMLP and HRBF for attitude control.

PubMed

Fortuna, L; Muscato, G; Xibilia, M G

2001-01-01

In this paper the problem of controlling the attitude of a rigid body, such as a Spacecraft, in three-dimensional space is approached by introducing two new control strategies developed in hypercomplex algebra. The proposed approaches are based on two parallel controllers, both derived in quaternion algebra. The first is a feedback controller of the proportional derivative (PD) type, while the second is a feedforward controller, which is implemented either by means of a hypercomplex multilayer perceptron (HMLP) neural network or by means of a hypercomplex radial basis function (HRBF) neural network. Several simulations show the performance of the two approaches. The results are also compared with a classical PD controller and with an adaptive controller, showing the improvements obtained by using neural networks, especially when an external disturbance acts on the rigid body. In particular the HMLP network gave better results when considering trajectories not presented during the learning phase.

Modeling the low-velocity impact characteristics of woven glass epoxy composite laminates using artificial neural networks

NASA Astrophysics Data System (ADS)

Mathivanan, N. Rajesh; Mouli, Chandra

2012-12-01

In this work, a new methodology based on artificial neural networks (ANN) has been developed to study the low-velocity impact characteristics of woven glass epoxy laminates of EP3 grade. To train and test the networks, multiple impact cases have been generated using statistical analysis of variance (ANOVA). Experimental tests were performed using an instrumented falling-weight impact-testing machine. Different impact velocities and impact energies on different thicknesses of laminates were considered as the input parameters of the ANN model. This model is a feed-forward back-propagation neural network. Using the input/output data of the experiments, the model was trained and tested. Further, the effects of the low-velocity impact response of the laminates at different energy levels were investigated by studying the cause-effect relationship among the influential factors using response surface methodology. The most significant parameter is determined from the other input variables through ANOVA.

A neural network controller for automated composite manufacturing

NASA Technical Reports Server (NTRS)

Lichtenwalner, Peter F.

1994-01-01

At McDonnell Douglas Aerospace (MDA), an artificial neural network based control system has been developed and implemented to control laser heating for the fiber placement composite manufacturing process. This neurocontroller learns an approximate inverse model of the process on-line to provide performance that improves with experience and exceeds that of conventional feedback control techniques. When untrained, the control system behaves as a proportional plus integral (PI) controller. However after learning from experience, the neural network feedforward control module provides control signals that greatly improve temperature tracking performance. Faster convergence to new temperature set points and reduced temperature deviation due to changing feed rate have been demonstrated on the machine. A Cerebellar Model Articulation Controller (CMAC) network is used for inverse modeling because of its rapid learning performance. This control system is implemented in an IBM compatible 386 PC with an A/D board interface to the machine.

A New Stochastic Technique for Painlevé Equation-I Using Neural Network Optimized with Swarm Intelligence

PubMed Central

Raja, Muhammad Asif Zahoor; Khan, Junaid Ali; Ahmad, Siraj-ul-Islam; Qureshi, Ijaz Mansoor

2012-01-01

A methodology for solution of Painlevé equation-I is presented using computational intelligence technique based on neural networks and particle swarm optimization hybridized with active set algorithm. The mathematical model of the equation is developed with the help of linear combination of feed-forward artificial neural networks that define the unsupervised error of the model. This error is minimized subject to the availability of appropriate weights of the networks. The learning of the weights is carried out using particle swarm optimization algorithm used as a tool for viable global search method, hybridized with active set algorithm for rapid local convergence. The accuracy, convergence rate, and computational complexity of the scheme are analyzed based on large number of independents runs and their comprehensive statistical analysis. The comparative studies of the results obtained are made with MATHEMATICA solutions, as well as, with variational iteration method and homotopy perturbation method. PMID:22919371

Modeling and control of magnetorheological fluid dampers using neural networks

NASA Astrophysics Data System (ADS)

Wang, D. H.; Liao, W. H.

2005-02-01

Due to the inherent nonlinear nature of magnetorheological (MR) fluid dampers, one of the challenging aspects for utilizing these devices to achieve high system performance is the development of accurate models and control algorithms that can take advantage of their unique characteristics. In this paper, the direct identification and inverse dynamic modeling for MR fluid dampers using feedforward and recurrent neural networks are studied. The trained direct identification neural network model can be used to predict the damping force of the MR fluid damper on line, on the basis of the dynamic responses across the MR fluid damper and the command voltage, and the inverse dynamic neural network model can be used to generate the command voltage according to the desired damping force through supervised learning. The architectures and the learning methods of the dynamic neural network models and inverse neural network models for MR fluid dampers are presented, and some simulation results are discussed. Finally, the trained neural network models are applied to predict and control the damping force of the MR fluid damper. Moreover, validation methods for the neural network models developed are proposed and used to evaluate their performance. Validation results with different data sets indicate that the proposed direct identification dynamic model using the recurrent neural network can be used to predict the damping force accurately and the inverse identification dynamic model using the recurrent neural network can act as a damper controller to generate the command voltage when the MR fluid damper is used in a semi-active mode.

Cascading a systolic array and a feedforward neural network for navigation and obstacle avoidance using potential fields

NASA Technical Reports Server (NTRS)

Plumer, Edward S.

1991-01-01

A technique is developed for vehicle navigation and control in the presence of obstacles. A potential function was devised that peaks at the surface of obstacles and has its minimum at the proper vehicle destination. This function is computed using a systolic array and is guaranteed not to have local minima. A feedfoward neural network is then used to control the steering of the vehicle using local potential field information. In this case, the vehicle is a trailer truck backing up. Previous work has demonstrated the capability of a neural network to control steering of such a trailer truck backing to a loading platform, but without obstacles. Now, the neural network was able to learn to navigate a trailer truck around obstacles while backing toward its destination. The network is trained in an obstacle free space to follow the negative gradient of the field, after which the network is able to control and navigate the truck to its target destination in a space of obstacles which may be stationary or movable.

Analysis and Modeling of DIII-D Experiments With OMFIT and Neural Networks

NASA Astrophysics Data System (ADS)

Meneghini, O.; Luna, C.; Smith, S. P.; Lao, L. L.; GA Theory Team

2013-10-01

The OMFIT integrated modeling framework is designed to facilitate experimental data analysis and enable integrated simulations. This talk introduces this framework and presents a selection of its applications to the DIII-D experiment. Examples include kinetic equilibrium reconstruction analysis; evaluation of MHD stability in the core and in the edge; and self-consistent predictive steady-state transport modeling. The OMFIT framework also provides the platform for an innovative approach based on neural networks to predict electron and ion energy fluxes. In our study a multi-layer feed-forward back-propagation neural network is built and trained over a database of DIII-D data. It is found that given the same parameters that the highest fidelity models use, the neural network model is able to predict to a large degree the heat transport profiles observed in the DIII-D experiments. Once the network is built, the numerical cost of evaluating the transport coefficients is virtually nonexistent, thus making the neural network model particularly well suited for plasma control and quick exploration of operational scenarios. The implementation of the neural network model and benchmark with experimental results and gyro-kinetic models will be discussed. Work supported in part by the US DOE under DE-FG02-95ER54309.

Sensorless control for permanent magnet synchronous motor using a neural network based adaptive estimator

NASA Astrophysics Data System (ADS)

Kwon, Chung-Jin; Kim, Sung-Joong; Han, Woo-Young; Min, Won-Kyoung

2005-12-01

The rotor position and speed estimation of permanent-magnet synchronous motor(PMSM) was dealt with. By measuring the phase voltages and currents of the PMSM drive, two diagonally recurrent neural network(DRNN) based observers, a neural current observer and a neural velocity observer were developed. DRNN which has self-feedback of the hidden neurons ensures that the outputs of DRNN contain the whole past information of the system even if the inputs of DRNN are only the present states and inputs of the system. Thus the structure of DRNN may be simpler than that of feedforward and fully recurrent neural networks. If the backpropagation method was used for the training of the DRNN the problem of slow convergence arise. In order to reduce this problem, recursive prediction error(RPE) based learning method for the DRNN was presented. The simulation results show that the proposed approach gives a good estimation of rotor speed and position, and RPE based training has requires a shorter computation time compared to backpropagation based training.

Computational properties of networks of synchronous groups of spiking neurons.

PubMed

Dayhoff, Judith E

2007-09-01

We demonstrate a model in which synchronously firing ensembles of neurons are networked to produce computational results. Each ensemble is a group of biological integrate-and-fire spiking neurons, with probabilistic interconnections between groups. An analogy is drawn in which each individual processing unit of an artificial neural network corresponds to a neuronal group in a biological model. The activation value of a unit in the artificial neural network corresponds to the fraction of active neurons, synchronously firing, in a biological neuronal group. Weights of the artificial neural network correspond to the product of the interconnection density between groups, the group size of the presynaptic group, and the postsynaptic potential heights in the synchronous group model. All three of these parameters can modulate connection strengths between neuronal groups in the synchronous group models. We give an example of nonlinear classification (XOR) and a function approximation example in which the capability of the artificial neural network can be captured by a neural network model with biological integrate-and-fire neurons configured as a network of synchronously firing ensembles of such neurons. We point out that the general function approximation capability proven for feedforward artificial neural networks appears to be approximated by networks of neuronal groups that fire in synchrony, where the groups comprise integrate-and-fire neurons. We discuss the advantages of this type of model for biological systems, its possible learning mechanisms, and the associated timing relationships.

Neural networks with fuzzy Petri nets for modeling a machining process

NASA Astrophysics Data System (ADS)

Hanna, Moheb M.

1998-03-01

The paper presents an intelligent architecture based a feedforward neural network with fuzzy Petri nets for modeling product quality in a CNC machining center. It discusses how the proposed architecture can be used for modeling, monitoring and control a product quality specification such as surface roughness. The surface roughness represents the output quality specification manufactured by a CNC machining center as a result of a milling process. The neural network approach employed the selected input parameters which defined by the machine operator via the CNC code. The fuzzy Petri nets approach utilized the exact input milling parameters, such as spindle speed, feed rate, tool diameter and coolant (off/on), which can be obtained via the machine or sensors system. An aim of the proposed architecture is to model the demanded quality of surface roughness as high, medium or low.

Dropout Prediction in E-Learning Courses through the Combination of Machine Learning Techniques

ERIC Educational Resources Information Center

Lykourentzou, Ioanna; Giannoukos, Ioannis; Nikolopoulos, Vassilis; Mpardis, George; Loumos, Vassili

2009-01-01

In this paper, a dropout prediction method for e-learning courses, based on three popular machine learning techniques and detailed student data, is proposed. The machine learning techniques used are feed-forward neural networks, support vector machines and probabilistic ensemble simplified fuzzy ARTMAP. Since a single technique may fail to…

Where Do Features Come From?

ERIC Educational Resources Information Center

Hinton, Geoffrey

2014-01-01

It is possible to learn multiple layers of non-linear features by backpropagating error derivatives through a feedforward neural network. This is a very effective learning procedure when there is a huge amount of labeled training data, but for many learning tasks very few labeled examples are available. In an effort to overcome the need for…

Feed-forward and generalized regression neural networks in modeling feeding behavior of pigs in the grow-finishing phase

USDA-ARS?s Scientific Manuscript database

Feeding patterns in group-housed grow-finishing pigs have been investigated for use in management decisions, identifying sick animals, and determining genetic differences within a herd. Development of models to predict swine feeding behaviour has been limited due the large number of potential enviro...

Feature Extraction and Classification of FLIR Imagery Using Relative Locations of Non-Homogeneous Regions with Feedforward Neural Networks

DTIC Science & Technology

1992-04-01

Proceedings of Tri-Service Data Fusion Symposium, Johns Hopkins University, May 1989. 39. F. Rosenblatt. Principles of Neurodynamics : Perceptrons and the...104 47. David E. Rummelhart and James L. McClelland. Parallel Distributed Processing: Explorations in the Microstructure of Cognition , volume 1. The

Modeling of the radiation belt megnetosphere in decisional timeframes

DOEpatents

Koller, Josef; Reeves, Geoffrey D; Friedel, Reiner H.W.

2013-04-23

Systems and methods for calculating L* in the magnetosphere with essentially the same accuracy as with a physics based model at many times the speed by developing a surrogate trained to be a surrogate for the physics-based model. The trained model can then beneficially process input data falling within the training range of the surrogate model. The surrogate model can be a feedforward neural network and the physics-based model can be the TSK03 model. Operatively, the surrogate model can use parameters on which the physics-based model was based, and/or spatial data for the location where L* is to be calculated. Surrogate models should be provided for each of a plurality of pitch angles. Accordingly, a surrogate model having a closed drift shell can be used from the plurality of models. The feedforward neural network can have a plurality of input-layer units, there being at least one input-layer unit for each physics-based model parameter, a plurality of hidden layer units and at least one output unit for the value of L*.

Feedforward neural network model estimating pollutant removal process within mesophilic upflow anaerobic sludge blanket bioreactor treating industrial starch processing wastewater.

PubMed

Antwi, Philip; Li, Jianzheng; Meng, Jia; Deng, Kaiwen; Koblah Quashie, Frank; Li, Jiuling; Opoku Boadi, Portia

2018-06-01

In this a, three-layered feedforward-backpropagation artificial neural network (BPANN) model was developed and employed to evaluate COD removal an upflow anaerobic sludge blanket (UASB) reactor treating industrial starch processing wastewater. At the end of UASB operation, microbial community characterization revealed satisfactory composition of microbes whereas morphology depicted rod-shaped archaea. pH, COD, NH 4 + , VFA, OLR and biogas yield were selected by principal component analysis and used as input variables. Whilst tangent sigmoid function (tansig) and linear function (purelin) were assigned as activation functions at the hidden-layer and output-layer, respectively, optimum BPANN architecture was achieved with Levenberg-Marquardt algorithm (trainlm) after eleven training algorithms had been tested. Based on performance indicators such the mean squared errors, fractional variance, index of agreement and coefficient of determination (R 2 ), the BPANN model demonstrated significant performance with R 2 reaching 87%. The study revealed that, control and optimization of an anaerobic digestion process with BPANN model was feasible. Copyright © 2018 Elsevier Ltd. All rights reserved.

A dynamic feedforward neural network based on gaussian particle swarm optimization and its application for predictive control.

PubMed

Han, Min; Fan, Jianchao; Wang, Jun

2011-09-01

A dynamic feedforward neural network (DFNN) is proposed for predictive control, whose adaptive parameters are adjusted by using Gaussian particle swarm optimization (GPSO) in the training process. Adaptive time-delay operators are added in the DFNN to improve its generalization for poorly known nonlinear dynamic systems with long time delays. Furthermore, GPSO adopts a chaotic map with Gaussian function to balance the exploration and exploitation capabilities of particles, which improves the computational efficiency without compromising the performance of the DFNN. The stability of the particle dynamics is analyzed, based on the robust stability theory, without any restrictive assumption. A stability condition for the GPSO+DFNN model is derived, which ensures a satisfactory global search and quick convergence, without the need for gradients. The particle velocity ranges could change adaptively during the optimization process. The results of a comparative study show that the performance of the proposed algorithm can compete with selected algorithms on benchmark problems. Additional simulation results demonstrate the effectiveness and accuracy of the proposed combination algorithm in identifying and controlling nonlinear systems with long time delays.

Expanding the occupational health methodology: A concatenated artificial neural network approach to model the burnout process in Chinese nurses.

PubMed

Ladstätter, Felix; Garrosa, Eva; Moreno-Jiménez, Bernardo; Ponsoda, Vicente; Reales Aviles, José Manuel; Dai, Junming

2016-01-01

Artificial neural networks are sophisticated modelling and prediction tools capable of extracting complex, non-linear relationships between predictor (input) and predicted (output) variables. This study explores this capacity by modelling non-linearities in the hardiness-modulated burnout process with a neural network. Specifically, two multi-layer feed-forward artificial neural networks are concatenated in an attempt to model the composite non-linear burnout process. Sensitivity analysis, a Monte Carlo-based global simulation technique, is then utilised to examine the first-order effects of the predictor variables on the burnout sub-dimensions and consequences. Results show that (1) this concatenated artificial neural network approach is feasible to model the burnout process, (2) sensitivity analysis is a prolific method to study the relative importance of predictor variables and (3) the relationships among variables involved in the development of burnout and its consequences are to different degrees non-linear. Many relationships among variables (e.g., stressors and strains) are not linear, yet researchers use linear methods such as Pearson correlation or linear regression to analyse these relationships. Artificial neural network analysis is an innovative method to analyse non-linear relationships and in combination with sensitivity analysis superior to linear methods.

Neural Network Compensation for Frequency Cross-Talk in Laser Interferometry

NASA Astrophysics Data System (ADS)

Lee, Wooram; Heo, Gunhaeng; You, Kwanho

The heterodyne laser interferometer acts as an ultra-precise measurement apparatus in semiconductor manufacture. However the periodical nonlinearity property caused from frequency cross-talk is an obstacle to improve the high measurement accuracy in nanometer scale. In order to minimize the nonlinearity error of the heterodyne interferometer, we propose a frequency cross-talk compensation algorithm using an artificial intelligence method. The feedforward neural network trained by back-propagation compensates the nonlinearity error and regulates to minimize the difference with the reference signal. With some experimental results, the improved accuracy is proved through comparison with the position value from a capacitive displacement sensor.

Learning in Stochastic Bit Stream Neural Networks.

PubMed

van Daalen, Max; Shawe-Taylor, John; Zhao, Jieyu

1996-08-01

This paper presents learning techniques for a novel feedforward stochastic neural network. The model uses stochastic weights and the "bit stream" data representation. It has a clean analysable functionality and is very attractive with its great potential to be implemented in hardware using standard digital VLSI technology. The design allows simulation at three different levels and learning techniques are described for each level. The lowest level corresponds to on-chip learning. Simulation results on three benchmark MONK's problems and handwritten digit recognition with a clean set of 500 16 x 16 pixel digits demonstrate that the new model is powerful enough for the real world applications. Copyright 1996 Elsevier Science Ltd

Polar cloud and surface classification using AVHRR imagery - An intercomparison of methods

NASA Technical Reports Server (NTRS)

Welch, R. M.; Sengupta, S. K.; Goroch, A. K.; Rabindra, P.; Rangaraj, N.; Navar, M. S.

1992-01-01

Six Advanced Very High-Resolution Radiometer local area coverage (AVHRR LAC) arctic scenes are classified into ten classes. Three different classifiers are examined: (1) the traditional stepwise discriminant analysis (SDA) method; (2) the feed-forward back-propagation (FFBP) neural network; and (3) the probabilistic neural network (PNN). More than 200 spectral and textural measures are computed. These are reduced to 20 features using sequential forward selection. Theoretical accuracy of the classifiers is determined using the bootstrap approach. Overall accuracy is 85.6 percent, 87.6 percent, and 87.0 percent for the SDA, FFBP, and PNN classifiers, respectively, with standard deviations of approximately 1 percent.

Efficiently modeling neural networks on massively parallel computers

NASA Technical Reports Server (NTRS)

Farber, Robert M.

1993-01-01

Neural networks are a very useful tool for analyzing and modeling complex real world systems. Applying neural network simulations to real world problems generally involves large amounts of data and massive amounts of computation. To efficiently handle the computational requirements of large problems, we have implemented at Los Alamos a highly efficient neural network compiler for serial computers, vector computers, vector parallel computers, and fine grain SIMD computers such as the CM-2 connection machine. This paper describes the mapping used by the compiler to implement feed-forward backpropagation neural networks for a SIMD (Single Instruction Multiple Data) architecture parallel computer. Thinking Machines Corporation has benchmarked our code at 1.3 billion interconnects per second (approximately 3 gigaflops) on a 64,000 processor CM-2 connection machine (Singer 1990). This mapping is applicable to other SIMD computers and can be implemented on MIMD computers such as the CM-5 connection machine. Our mapping has virtually no communications overhead with the exception of the communications required for a global summation across the processors (which has a sub-linear runtime growth on the order of O(log(number of processors)). We can efficiently model very large neural networks which have many neurons and interconnects and our mapping can extend to arbitrarily large networks (within memory limitations) by merging the memory space of separate processors with fast adjacent processor interprocessor communications. This paper will consider the simulation of only feed forward neural network although this method is extendable to recurrent networks.

Integration of Online Parameter Identification and Neural Network for In-Flight Adaptive Control

NASA Technical Reports Server (NTRS)

Hageman, Jacob J.; Smith, Mark S.; Stachowiak, Susan

2003-01-01

An indirect adaptive system has been constructed for robust control of an aircraft with uncertain aerodynamic characteristics. This system consists of a multilayer perceptron pre-trained neural network, online stability and control derivative identification, a dynamic cell structure online learning neural network, and a model following control system based on the stochastic optimal feedforward and feedback technique. The pre-trained neural network and model following control system have been flight-tested, but the online parameter identification and online learning neural network are new additions used for in-flight adaptation of the control system model. A description of the modification and integration of these two stand-alone software packages into the complete system in preparation for initial flight tests is presented. Open-loop results using both simulation and flight data, as well as closed-loop performance of the complete system in a nonlinear, six-degree-of-freedom, flight validated simulation, are analyzed. Results show that this online learning system, in contrast to the nonlearning system, has the ability to adapt to changes in aerodynamic characteristics in a real-time, closed-loop, piloted simulation, resulting in improved flying qualities.

On the complexity of neural network classifiers: a comparison between shallow and deep architectures.

PubMed

Bianchini, Monica; Scarselli, Franco

2014-08-01

Recently, researchers in the artificial neural network field have focused their attention on connectionist models composed by several hidden layers. In fact, experimental results and heuristic considerations suggest that deep architectures are more suitable than shallow ones for modern applications, facing very complex problems, e.g., vision and human language understanding. However, the actual theoretical results supporting such a claim are still few and incomplete. In this paper, we propose a new approach to study how the depth of feedforward neural networks impacts on their ability in implementing high complexity functions. First, a new measure based on topological concepts is introduced, aimed at evaluating the complexity of the function implemented by a neural network, used for classification purposes. Then, deep and shallow neural architectures with common sigmoidal activation functions are compared, by deriving upper and lower bounds on their complexity, and studying how the complexity depends on the number of hidden units and the used activation function. The obtained results seem to support the idea that deep networks actually implements functions of higher complexity, so that they are able, with the same number of resources, to address more difficult problems.

Forecasting the portuguese stock market time series by using artificial neural networks

NASA Astrophysics Data System (ADS)

Isfan, Monica; Menezes, Rui; Mendes, Diana A.

2010-04-01

In this paper, we show that neural networks can be used to uncover the non-linearity that exists in the financial data. First, we follow a traditional approach by analysing the deterministic/stochastic characteristics of the Portuguese stock market data and some typical features are studied, like the Hurst exponents, among others. We also simulate a BDS test to investigate nonlinearities and the results are as expected: the financial time series do not exhibit linear dependence. Secondly, we trained four types of neural networks for the stock markets and used the models to make forecasts. The artificial neural networks were obtained using a three-layer feed-forward topology and the back-propagation learning algorithm. The quite large number of parameters that must be selected to develop a neural network forecasting model involves some trial and as a consequence the error is not small enough. In order to improve this we use a nonlinear optimization algorithm to minimize the error. Finally, the output of the 4 models is quite similar, leading to a qualitative forecast that we compare with the results of the application of k-nearest-neighbor for the same time series.

Smooth function approximation using neural networks.

PubMed

Ferrari, Silvia; Stengel, Robert F

2005-01-01

An algebraic approach for representing multidimensional nonlinear functions by feedforward neural networks is presented. In this paper, the approach is implemented for the approximation of smooth batch data containing the function's input, output, and possibly, gradient information. The training set is associated to the network adjustable parameters by nonlinear weight equations. The cascade structure of these equations reveals that they can be treated as sets of linear systems. Hence, the training process and the network approximation properties can be investigated via linear algebra. Four algorithms are developed to achieve exact or approximate matching of input-output and/or gradient-based training sets. Their application to the design of forward and feedback neurocontrollers shows that algebraic training is characterized by faster execution speeds and better generalization properties than contemporary optimization techniques.

Self-organization of synchronous activity propagation in neuronal networks driven by local excitation

PubMed Central

Bayati, Mehdi; Valizadeh, Alireza; Abbassian, Abdolhossein; Cheng, Sen

2015-01-01

Many experimental and theoretical studies have suggested that the reliable propagation of synchronous neural activity is crucial for neural information processing. The propagation of synchronous firing activity in so-called synfire chains has been studied extensively in feed-forward networks of spiking neurons. However, it remains unclear how such neural activity could emerge in recurrent neuronal networks through synaptic plasticity. In this study, we investigate whether local excitation, i.e., neurons that fire at a higher frequency than the other, spontaneously active neurons in the network, can shape a network to allow for synchronous activity propagation. We use two-dimensional, locally connected and heterogeneous neuronal networks with spike-timing dependent plasticity (STDP). We find that, in our model, local excitation drives profound network changes within seconds. In the emergent network, neural activity propagates synchronously through the network. This activity originates from the site of the local excitation and propagates through the network. The synchronous activity propagation persists, even when the local excitation is removed, since it derives from the synaptic weight matrix. Importantly, once this connectivity is established it remains stable even in the presence of spontaneous activity. Our results suggest that synfire-chain-like activity can emerge in a relatively simple way in realistic neural networks by locally exciting the desired origin of the neuronal sequence. PMID:26089794

Computational modeling of spiking neural network with learning rules from STDP and intrinsic plasticity

NASA Astrophysics Data System (ADS)

Li, Xiumin; Wang, Wei; Xue, Fangzheng; Song, Yongduan

2018-02-01

Recently there has been continuously increasing interest in building up computational models of spiking neural networks (SNN), such as the Liquid State Machine (LSM). The biologically inspired self-organized neural networks with neural plasticity can enhance the capability of computational performance, with the characteristic features of dynamical memory and recurrent connection cycles which distinguish them from the more widely used feedforward neural networks. Despite a variety of computational models for brain-like learning and information processing have been proposed, the modeling of self-organized neural networks with multi-neural plasticity is still an important open challenge. The main difficulties lie in the interplay among different forms of neural plasticity rules and understanding how structures and dynamics of neural networks shape the computational performance. In this paper, we propose a novel approach to develop the models of LSM with a biologically inspired self-organizing network based on two neural plasticity learning rules. The connectivity among excitatory neurons is adapted by spike-timing-dependent plasticity (STDP) learning; meanwhile, the degrees of neuronal excitability are regulated to maintain a moderate average activity level by another learning rule: intrinsic plasticity (IP). Our study shows that LSM with STDP+IP performs better than LSM with a random SNN or SNN obtained by STDP alone. The noticeable improvement with the proposed method is due to the better reflected competition among different neurons in the developed SNN model, as well as the more effectively encoded and processed relevant dynamic information with its learning and self-organizing mechanism. This result gives insights to the optimization of computational models of spiking neural networks with neural plasticity.

Visual crowding illustrates the inadequacy of local vs. global and feedforward vs. feedback distinctions in modeling visual perception

PubMed Central

Clarke, Aaron M.; Herzog, Michael H.; Francis, Gregory

2014-01-01

Experimentalists tend to classify models of visual perception as being either local or global, and involving either feedforward or feedback processing. We argue that these distinctions are not as helpful as they might appear, and we illustrate these issues by analyzing models of visual crowding as an example. Recent studies have argued that crowding cannot be explained by purely local processing, but that instead, global factors such as perceptual grouping are crucial. Theories of perceptual grouping, in turn, often invoke feedback connections as a way to account for their global properties. We examined three types of crowding models that are representative of global processing models, and two of which employ feedback processing: a model based on Fourier filtering, a feedback neural network, and a specific feedback neural architecture that explicitly models perceptual grouping. Simulations demonstrate that crucial empirical findings are not accounted for by any of the models. We conclude that empirical investigations that reject a local or feedforward architecture offer almost no constraints for model construction, as there are an uncountable number of global and feedback systems. We propose that the identification of a system as being local or global and feedforward or feedback is less important than the identification of a system's computational details. Only the latter information can provide constraints on model development and promote quantitative explanations of complex phenomena. PMID:25374554

Holographic neural networks versus conventional neural networks: a comparative evaluation for the classification of landmine targets in ground-penetrating radar images

NASA Astrophysics Data System (ADS)

Mudigonda, Naga R.; Kacelenga, Ray; Edwards, Mark

2004-09-01

This paper evaluates the performance of a holographic neural network in comparison with a conventional feedforward backpropagation neural network for the classification of landmine targets in ground penetrating radar images. The data used in the study was acquired from four different test sites using the landmine detection system developed by General Dynamics Canada Ltd., in collaboration with the Defense Research and Development Canada, Suffield. A set of seven features extracted for each detected alarm is used as stimulus inputs for the networks. The recall responses of the networks are then evaluated against the ground truth to declare true or false detections. The area computed under the receiver operating characteristic curve is used for comparative purposes. With a large dataset comprising of data from multiple sites, both the holographic and conventional networks showed comparable trends in recall accuracies with area values of 0.88 and 0.87, respectively. By using independent validation datasets, the holographic network"s generalization performance was observed to be better (mean area = 0.86) as compared to the conventional network (mean area = 0.82). Despite the widely publicized theoretical advantages of the holographic technology, use of more than the required number of cortical memory elements resulted in an over-fitting phenomenon of the holographic network.

Application of machine learning methods for traffic signs recognition

NASA Astrophysics Data System (ADS)

Filatov, D. V.; Ignatev, K. V.; Deviatkin, A. V.; Serykh, E. V.

2018-02-01

This paper focuses on solving a relevant and pressing safety issue on intercity roads. Two approaches were considered for solving the problem of traffic signs recognition; the approaches involved neural networks to analyze images obtained from a camera in the real-time mode. The first approach is based on a sequential image processing. At the initial stage, with the help of color filters and morphological operations (dilatation and erosion), the area containing the traffic sign is located on the image, then the selected and scaled fragment of the image is analyzed using a feedforward neural network to determine the meaning of the found traffic sign. Learning of the neural network in this approach is carried out using a backpropagation method. The second approach involves convolution neural networks at both stages, i.e. when searching and selecting the area of the image containing the traffic sign, and when determining its meaning. Learning of the neural network in the second approach is carried out using the intersection over union function and a loss function. For neural networks to learn and the proposed algorithms to be tested, a series of videos from a dash cam were used that were shot under various weather and illumination conditions. As a result, the proposed approaches for traffic signs recognition were analyzed and compared by key indicators such as recognition rate percentage and the complexity of neural networks’ learning process.

The modelling of lead removal from water by deep eutectic solvents functionalized CNTs: artificial neural network (ANN) approach.

PubMed

Fiyadh, Seef Saadi; AlSaadi, Mohammed Abdulhakim; AlOmar, Mohamed Khalid; Fayaed, Sabah Saadi; Hama, Ako R; Bee, Sharifah; El-Shafie, Ahmed

2017-11-01

The main challenge in the lead removal simulation is the behaviour of non-linearity relationships between the process parameters. The conventional modelling technique usually deals with this problem by a linear method. The substitute modelling technique is an artificial neural network (ANN) system, and it is selected to reflect the non-linearity in the interaction among the variables in the function. Herein, synthesized deep eutectic solvents were used as a functionalized agent with carbon nanotubes as adsorbents of Pb 2+ . Different parameters were used in the adsorption study including pH (2.7 to 7), adsorbent dosage (5 to 20 mg), contact time (3 to 900 min) and Pb 2+ initial concentration (3 to 60 mg/l). The number of experimental trials to feed and train the system was 158 runs conveyed in laboratory scale. Two ANN types were designed in this work, the feed-forward back-propagation and layer recurrent; both methods are compared based on their predictive proficiency in terms of the mean square error (MSE), root mean square error, relative root mean square error, mean absolute percentage error and determination coefficient (R 2 ) based on the testing dataset. The ANN model of lead removal was subjected to accuracy determination and the results showed R 2 of 0.9956 with MSE of 1.66 × 10 -4 . The maximum relative error is 14.93% for the feed-forward back-propagation neural network model.

Optimization of a Multi-Stage ATR System for Small Target Identification

NASA Technical Reports Server (NTRS)

Lin, Tsung-Han; Lu, Thomas; Braun, Henry; Edens, Western; Zhang, Yuhan; Chao, Tien- Hsin; Assad, Christopher; Huntsberger, Terrance

2010-01-01

An Automated Target Recognition system (ATR) was developed to locate and target small object in images and videos. The data is preprocessed and sent to a grayscale optical correlator (GOC) filter to identify possible regionsof- interest (ROIs). Next, features are extracted from ROIs based on Principal Component Analysis (PCA) and sent to neural network (NN) to be classified. The features are analyzed by the NN classifier indicating if each ROI contains the desired target or not. The ATR system was found useful in identifying small boats in open sea. However, due to "noisy background," such as weather conditions, background buildings, or water wakes, some false targets are mis-classified. Feedforward backpropagation and Radial Basis neural networks are optimized for generalization of representative features to reduce false-alarm rate. The neural networks are compared for their performance in classification accuracy, classifying time, and training time.

Adaptive online inverse control of a shape memory alloy wire actuator using a dynamic neural network

NASA Astrophysics Data System (ADS)

Mai, Huanhuan; Song, Gangbing; Liao, Xiaofeng

2013-01-01

Shape memory alloy (SMA) actuators exhibit severe hysteresis, a nonlinear behavior, which complicates control strategies and limits their applications. This paper presents a new approach to controlling an SMA actuator through an adaptive inverse model based controller that consists of a dynamic neural network (DNN) identifier, a copy dynamic neural network (CDNN) feedforward term and a proportional (P) feedback action. Unlike fixed hysteresis models used in most inverse controllers, the proposed one uses a DNN to identify online the relationship between the applied voltage to the actuator and the displacement (the inverse model). Even without a priori knowledge of the SMA hysteresis and without pre-training, the proposed controller can precisely control the SMA wire actuator in various tracking tasks by identifying online the inverse model of the SMA actuator. Experiments were conducted, and experimental results demonstrated real-time modeling capabilities of DNN and the performance of the adaptive inverse controller.

Constitutive flow behaviour of austenitic stainless steels under hot deformation: artificial neural network modelling to understand, evaluate and predict

NASA Astrophysics Data System (ADS)

Mandal, Sumantra; Sivaprasad, P. V.; Venugopal, S.; Murthy, K. P. N.

2006-09-01

An artificial neural network (ANN) model is developed to predict the constitutive flow behaviour of austenitic stainless steels during hot deformation. The input parameters are alloy composition and process variables whereas flow stress is the output. The model is based on a three-layer feed-forward ANN with a back-propagation learning algorithm. The neural network is trained with an in-house database obtained from hot compression tests on various grades of austenitic stainless steels. The performance of the model is evaluated using a wide variety of statistical indices. Good agreement between experimental and predicted data is obtained. The correlation between individual alloying elements and high temperature flow behaviour is investigated by employing the ANN model. The results are found to be consistent with the physical phenomena. The model can be used as a guideline for new alloy development.

Approximating quantum many-body wave functions using artificial neural networks

NASA Astrophysics Data System (ADS)

Cai, Zi; Liu, Jinguo

2018-01-01

In this paper, we demonstrate the expressibility of artificial neural networks (ANNs) in quantum many-body physics by showing that a feed-forward neural network with a small number of hidden layers can be trained to approximate with high precision the ground states of some notable quantum many-body systems. We consider the one-dimensional free bosons and fermions, spinless fermions on a square lattice away from half-filling, as well as frustrated quantum magnetism with a rapidly oscillating ground-state characteristic function. In the latter case, an ANN with a standard architecture fails, while that with a slightly modified one successfully learns the frustration-induced complex sign rule in the ground state and approximates the ground states with high precisions. As an example of practical use of our method, we also perform the variational method to explore the ground state of an antiferromagnetic J1-J2 Heisenberg model.

Role of intraglomerular circuits in shaping temporally structured responses to naturalistic inhalation-driven sensory input to the olfactory bulb

PubMed Central

Carey, Ryan M.; Sherwood, William Erik; Shipley, Michael T.; Borisyuk, Alla

2015-01-01

Olfaction in mammals is a dynamic process driven by the inhalation of air through the nasal cavity. Inhalation determines the temporal structure of sensory neuron responses and shapes the neural dynamics underlying central olfactory processing. Inhalation-linked bursts of activity among olfactory bulb (OB) output neurons [mitral/tufted cells (MCs)] are temporally transformed relative to those of sensory neurons. We investigated how OB circuits shape inhalation-driven dynamics in MCs using a modeling approach that was highly constrained by experimental results. First, we constructed models of canonical OB circuits that included mono- and disynaptic feedforward excitation, recurrent inhibition and feedforward inhibition of the MC. We then used experimental data to drive inputs to the models and to tune parameters; inputs were derived from sensory neuron responses during natural odorant sampling (sniffing) in awake rats, and model output was compared with recordings of MC responses to odorants sampled with the same sniff waveforms. This approach allowed us to identify OB circuit features underlying the temporal transformation of sensory inputs into inhalation-linked patterns of MC spike output. We found that realistic input-output transformations can be achieved independently by multiple circuits, including feedforward inhibition with slow onset and decay kinetics and parallel feedforward MC excitation mediated by external tufted cells. We also found that recurrent and feedforward inhibition had differential impacts on MC firing rates and on inhalation-linked response dynamics. These results highlight the importance of investigating neural circuits in a naturalistic context and provide a framework for further explorations of signal processing by OB networks. PMID:25717156

Designing an Adaptive Web-Based Learning System Based on Students' Cognitive Styles Identified Online

ERIC Educational Resources Information Center

Lo, Jia-Jiunn; Chan, Ya-Chen; Yeh, Shiou-Wen

2012-01-01

This study developed an adaptive web-based learning system focusing on students' cognitive styles. The system is composed of a student model and an adaptation model. It collected students' browsing behaviors to update the student model for unobtrusively identifying student cognitive styles through a multi-layer feed-forward neural network (MLFF).…

A feed-forward neural network for modeling feeding behavior of group-housed grow-finish pigs with respect to thermal conditions

USDA-ARS?s Scientific Manuscript database

Feeding patterns of pigs in the grow-finish phase have been investigated for use in management decisions, identifying sick animals, and determining genetic differences within a herd. Development of models to predict swine feeding behavior has been limited due the large number of potential environmen...

Connectomic constraints on computation in feedforward networks of spiking neurons.

PubMed

Ramaswamy, Venkatakrishnan; Banerjee, Arunava

2014-10-01

Several efforts are currently underway to decipher the connectome or parts thereof in a variety of organisms. Ascertaining the detailed physiological properties of all the neurons in these connectomes, however, is out of the scope of such projects. It is therefore unclear to what extent knowledge of the connectome alone will advance a mechanistic understanding of computation occurring in these neural circuits, especially when the high-level function of the said circuit is unknown. We consider, here, the question of how the wiring diagram of neurons imposes constraints on what neural circuits can compute, when we cannot assume detailed information on the physiological response properties of the neurons. We call such constraints-that arise by virtue of the connectome-connectomic constraints on computation. For feedforward networks equipped with neurons that obey a deterministic spiking neuron model which satisfies a small number of properties, we ask if just by knowing the architecture of a network, we can rule out computations that it could be doing, no matter what response properties each of its neurons may have. We show results of this form, for certain classes of network architectures. On the other hand, we also prove that with the limited set of properties assumed for our model neurons, there are fundamental limits to the constraints imposed by network structure. Thus, our theory suggests that while connectomic constraints might restrict the computational ability of certain classes of network architectures, we may require more elaborate information on the properties of neurons in the network, before we can discern such results for other classes of networks.

Network Receptive Field Modeling Reveals Extensive Integration and Multi-feature Selectivity in Auditory Cortical Neurons.

PubMed

Harper, Nicol S; Schoppe, Oliver; Willmore, Ben D B; Cui, Zhanfeng; Schnupp, Jan W H; King, Andrew J

2016-11-01

Cortical sensory neurons are commonly characterized using the receptive field, the linear dependence of their response on the stimulus. In primary auditory cortex neurons can be characterized by their spectrotemporal receptive fields, the spectral and temporal features of a sound that linearly drive a neuron. However, receptive fields do not capture the fact that the response of a cortical neuron results from the complex nonlinear network in which it is embedded. By fitting a nonlinear feedforward network model (a network receptive field) to cortical responses to natural sounds, we reveal that primary auditory cortical neurons are sensitive over a substantially larger spectrotemporal domain than is seen in their standard spectrotemporal receptive fields. Furthermore, the network receptive field, a parsimonious network consisting of 1-7 sub-receptive fields that interact nonlinearly, consistently better predicts neural responses to auditory stimuli than the standard receptive fields. The network receptive field reveals separate excitatory and inhibitory sub-fields with different nonlinear properties, and interaction of the sub-fields gives rise to important operations such as gain control and conjunctive feature detection. The conjunctive effects, where neurons respond only if several specific features are present together, enable increased selectivity for particular complex spectrotemporal structures, and may constitute an important stage in sound recognition. In conclusion, we demonstrate that fitting auditory cortical neural responses with feedforward network models expands on simple linear receptive field models in a manner that yields substantially improved predictive power and reveals key nonlinear aspects of cortical processing, while remaining easy to interpret in a physiological context.

Network Receptive Field Modeling Reveals Extensive Integration and Multi-feature Selectivity in Auditory Cortical Neurons

PubMed Central

Willmore, Ben D. B.; Cui, Zhanfeng; Schnupp, Jan W. H.; King, Andrew J.

2016-01-01

Cortical sensory neurons are commonly characterized using the receptive field, the linear dependence of their response on the stimulus. In primary auditory cortex neurons can be characterized by their spectrotemporal receptive fields, the spectral and temporal features of a sound that linearly drive a neuron. However, receptive fields do not capture the fact that the response of a cortical neuron results from the complex nonlinear network in which it is embedded. By fitting a nonlinear feedforward network model (a network receptive field) to cortical responses to natural sounds, we reveal that primary auditory cortical neurons are sensitive over a substantially larger spectrotemporal domain than is seen in their standard spectrotemporal receptive fields. Furthermore, the network receptive field, a parsimonious network consisting of 1–7 sub-receptive fields that interact nonlinearly, consistently better predicts neural responses to auditory stimuli than the standard receptive fields. The network receptive field reveals separate excitatory and inhibitory sub-fields with different nonlinear properties, and interaction of the sub-fields gives rise to important operations such as gain control and conjunctive feature detection. The conjunctive effects, where neurons respond only if several specific features are present together, enable increased selectivity for particular complex spectrotemporal structures, and may constitute an important stage in sound recognition. In conclusion, we demonstrate that fitting auditory cortical neural responses with feedforward network models expands on simple linear receptive field models in a manner that yields substantially improved predictive power and reveals key nonlinear aspects of cortical processing, while remaining easy to interpret in a physiological context. PMID:27835647

Neural network approach to the inverse problem of the crack-depth determination from ultrasonic backscattering data

NASA Astrophysics Data System (ADS)

Takadoya, M.; Notake, M.; Kitahara, M.; Achenbach, J. D.; Guo, Q. C.; Peterson, M. L.

A neural network approach has been developed to determine the depth of a surface breaking crack in a steel plate from ultrasonic backscattering data. The network is trained by the use of a feedforward three-layered network together with a back-propagation algorithm for error corrections. Synthetic data are employed for network training. The signal used for crack isonification is a mode converted 45 deg transverse wave. The plate with a surface breaking crack is immersed in water, and the crack is insonified from the opposite uncracked side of the plate. A numerical analysis of the backscattered field is carried out based on the elastic wave theory by the use of the boundary element method. The numerical analysis provides synthetic data for the training of the network. The training data have been calculated for cracks with specific increments of the experimental data which are different from the training data.

A neural network device for on-line particle identification in cosmic ray experiments

NASA Astrophysics Data System (ADS)

Scrimaglio, R.; Finetti, N.; D'Altorio, L.; Rantucci, E.; Raso, M.; Segreto, E.; Tassoni, A.; Cardarilli, G. C.

2004-05-01

On-line particle identification is one of the main goals of many experiments in space both for rare event studies and for optimizing measurements along the orbital trajectory. Neural networks can be a useful tool for signal processing and real time data analysis in such experiments. In this document we report on the performances of a programmable neural device which was developed in VLSI analog/digital technology. Neurons and synapses were accomplished by making use of Operational Transconductance Amplifier (OTA) structures. In this paper we report on the results of measurements performed in order to verify the agreement of the characteristic curves of each elementary cell with simulations and on the device performances obtained by implementing simple neural structures on the VLSI chip. A feed-forward neural network (Multi-Layer Perceptron, MLP) was implemented on the VLSI chip and trained to identify particles by processing the signals of two-dimensional position-sensitive Si detectors. The radiation monitoring device consisted of three double-sided silicon strip detectors. From the analysis of a set of simulated data it was found that the MLP implemented on the neural device gave results comparable with those obtained with the standard method of analysis confirming that the implemented neural network could be employed for real time particle identification.

Artificial neural network modelling of uncertainty in gamma-ray spectrometry

NASA Astrophysics Data System (ADS)

Dragović, S.; Onjia, A.; Stanković, S.; Aničin, I.; Bačić, G.

2005-03-01

An artificial neural network (ANN) model for the prediction of measuring uncertainties in gamma-ray spectrometry was developed and optimized. A three-layer feed-forward ANN with back-propagation learning algorithm was used to model uncertainties of measurement of activity levels of eight radionuclides ( 226Ra, 238U, 235U, 40K, 232Th, 134Cs, 137Cs and 7Be) in soil samples as a function of measurement time. It was shown that the neural network provides useful data even from small experimental databases. The performance of the optimized neural network was found to be very good, with correlation coefficients ( R2) between measured and predicted uncertainties ranging from 0.9050 to 0.9915. The correlation coefficients did not significantly deteriorate when the network was tested on samples with greatly different uranium-to-thorium ( 238U/ 232Th) ratios. The differences between measured and predicted uncertainties were not influenced by the absolute values of uncertainties of measured radionuclide activities. Once the ANN is trained, it could be employed in analyzing soil samples regardless of the 238U/ 232Th ratio. It was concluded that a considerable saving in time could be obtained using the trained neural network model for predicting the measurement times needed to attain the desired statistical accuracy.

End-to-End ASR-Free Keyword Search From Speech

NASA Astrophysics Data System (ADS)

Audhkhasi, Kartik; Rosenberg, Andrew; Sethy, Abhinav; Ramabhadran, Bhuvana; Kingsbury, Brian

2017-12-01

End-to-end (E2E) systems have achieved competitive results compared to conventional hybrid hidden Markov model (HMM)-deep neural network based automatic speech recognition (ASR) systems. Such E2E systems are attractive due to the lack of dependence on alignments between input acoustic and output grapheme or HMM state sequence during training. This paper explores the design of an ASR-free end-to-end system for text query-based keyword search (KWS) from speech trained with minimal supervision. Our E2E KWS system consists of three sub-systems. The first sub-system is a recurrent neural network (RNN)-based acoustic auto-encoder trained to reconstruct the audio through a finite-dimensional representation. The second sub-system is a character-level RNN language model using embeddings learned from a convolutional neural network. Since the acoustic and text query embeddings occupy different representation spaces, they are input to a third feed-forward neural network that predicts whether the query occurs in the acoustic utterance or not. This E2E ASR-free KWS system performs respectably despite lacking a conventional ASR system and trains much faster.

Neural Sequence Generation Using Spatiotemporal Patterns of Inhibition.

PubMed

Cannon, Jonathan; Kopell, Nancy; Gardner, Timothy; Markowitz, Jeffrey

2015-11-01

Stereotyped sequences of neural activity are thought to underlie reproducible behaviors and cognitive processes ranging from memory recall to arm movement. One of the most prominent theoretical models of neural sequence generation is the synfire chain, in which pulses of synchronized spiking activity propagate robustly along a chain of cells connected by highly redundant feedforward excitation. But recent experimental observations in the avian song production pathway during song generation have shown excitatory activity interacting strongly with the firing patterns of inhibitory neurons, suggesting a process of sequence generation more complex than feedforward excitation. Here we propose a model of sequence generation inspired by these observations in which a pulse travels along a spatially recurrent excitatory chain, passing repeatedly through zones of local feedback inhibition. In this model, synchrony and robust timing are maintained not through redundant excitatory connections, but rather through the interaction between the pulse and the spatiotemporal pattern of inhibition that it creates as it circulates the network. These results suggest that spatially and temporally structured inhibition may play a key role in sequence generation.

Reformulated Neural Network (ReNN): a New Alternative for Data-driven Modelling in Hydrology and Water Resources Engineering

NASA Astrophysics Data System (ADS)

Razavi, S.; Tolson, B.; Burn, D.; Seglenieks, F.

2012-04-01

Reformulated Neural Network (ReNN) has been recently developed as an efficient and more effective alternative to feedforward multi-layer perceptron (MLP) neural networks [Razavi, S., and Tolson, B. A. (2011). "A new formulation for feedforward neural networks." IEEE Transactions on Neural Networks, 22(10), 1588-1598, DOI: 1510.1109/TNN.2011.2163169]. This presentation initially aims to introduce the ReNN to the water resources community and then demonstrates ReNN applications to water resources related problems. ReNN is essentially equivalent to a single-hidden-layer MLP neural network but defined on a new set of network variables which is more effective than the traditional set of network weights and biases. The main features of the new network variables are that they are geometrically interpretable and each variable has a distinct role in forming the network response. ReNN is more efficiently trained as it has a less complex error response surface. In addition to the ReNN training efficiency, the interpretability of the ReNN variables enables the users to monitor and understand the internal behaviour of the network while training. Regularization in the ReNN response can be also directly measured and controlled. This feature improves the generalization ability of the network. The appeal of the ReNN is demonstrated with two ReNN applications to water resources engineering problems. In the first application, the ReNN is used to model the rainfall-runoff relationships in multiple watersheds in the Great Lakes basin located in northeastern North America. Modelling inflows to the Great Lakes are of great importance to the management of the Great Lakes system. Due to the lack of some detailed physical data about existing control structures in many subwatersheds of this huge basin, the data-driven approach to modelling such as the ReNN are required to replace predictions from a physically-based rainfall runoff model. Unlike traditional MLPs, the ReNN does not necessarily require an independent set of data for validation as the ReNN has the capability to control and verify the network degree of regularization. As such, the ReNN can be very beneficial in this case study as the data available in this case study is limited. In the second application, ReNN is fitted on the response function of the SWAT hydrologic model to act as a fast-to-run response surface surrogate (i.e., metamodel) of the original computationally intensive SWAT model. Besides the training efficiency gains, the ReNN applications demonstrate how the ReNN interpretability could help users develop more reliable networks which perform predictably better in terms of generalization.

Overreliance on auditory feedback may lead to sound/syllable repetitions: simulations of stuttering and fluency-inducing conditions with a neural model of speech production.

PubMed

Civier, Oren; Tasko, Stephen M; Guenther, Frank H

2010-09-01

This paper investigates the hypothesis that stuttering may result in part from impaired readout of feedforward control of speech, which forces persons who stutter (PWS) to produce speech with a motor strategy that is weighted too much toward auditory feedback control. Over-reliance on feedback control leads to production errors which if they grow large enough, can cause the motor system to "reset" and repeat the current syllable. This hypothesis is investigated using computer simulations of a "neurally impaired" version of the DIVA model, a neural network model of speech acquisition and production. The model's outputs are compared to published acoustic data from PWS' fluent speech, and to combined acoustic and articulatory movement data collected from the dysfluent speech of one PWS. The simulations mimic the errors observed in the PWS subject's speech, as well as the repairs of these errors. Additional simulations were able to account for enhancements of fluency gained by slowed/prolonged speech and masking noise. Together these results support the hypothesis that many dysfluencies in stuttering are due to a bias away from feedforward control and toward feedback control. The reader will be able to (a) describe the contribution of auditory feedback control and feedforward control to normal and stuttered speech production, (b) summarize the neural modeling approach to speech production and its application to stuttering, and (c) explain how the DIVA model accounts for enhancements of fluency gained by slowed/prolonged speech and masking noise.

The experimental identification of magnetorheological dampers and evaluation of their controllers

NASA Astrophysics Data System (ADS)

Metered, H.; Bonello, P.; Oyadiji, S. O.

2010-05-01

Magnetorheological (MR) fluid dampers are semi-active control devices that have been applied over a wide range of practical vibration control applications. This paper concerns the experimental identification of the dynamic behaviour of an MR damper and the use of the identified parameters in the control of such a damper. Feed-forward and recurrent neural networks are used to model both the direct and inverse dynamics of the damper. Training and validation of the proposed neural networks are achieved by using the data generated through dynamic tests with the damper mounted on a tensile testing machine. The validation test results clearly show that the proposed neural networks can reliably represent both the direct and inverse dynamic behaviours of an MR damper. The effect of the cylinder's surface temperature on both the direct and inverse dynamics of the damper is studied, and the neural network model is shown to be reasonably robust against significant temperature variation. The inverse recurrent neural network model is introduced as a damper controller and experimentally evaluated against alternative controllers proposed in the literature. The results reveal that the neural-based damper controller offers superior damper control. This observation and the added advantages of low-power requirement, extended service life of the damper and the minimal use of sensors, indicate that a neural-based damper controller potentially offers the most cost-effective vibration control solution among the controllers investigated.

Neural network pattern recognition of thermal-signature spectra for chemical defense

NASA Astrophysics Data System (ADS)

Carrieri, Arthur H.; Lim, Pascal I.

1995-05-01

We treat infrared patterns of absorption or emission by nerve and blister agent compounds (and simulants of this chemical group) as features for the training of neural networks to detect the compounds' liquid layers on the ground or their vapor plumes during evaporation by external heating. Training of a four-layer network architecture is composed of a backward-error-propagation algorithm and a gradient-descent paradigm. We conduct testing by feed-forwarding preprocessed spectra through the network in a scaled format consistent with the structure of the training-data-set representation. The best-performance weight matrix (spectral filter) evolved from final network training and testing with software simulation trials is electronically transferred to a set of eight artificial intelligence integrated circuits (ICs') in specific modular form (splitting of weight matrices). This form makes full use of all input-output IC nodes. This neural network computer serves an important real-time detection function when it is integrated into pre-and postprocessing data-handling units of a tactical prototype thermoluminescence sensor now under development at the Edgewood Research, Development, and Engineering Center.

Hardware-Efficient On-line Learning through Pipelined Truncated-Error Backpropagation in Binary-State Networks

PubMed Central

Mostafa, Hesham; Pedroni, Bruno; Sheik, Sadique; Cauwenberghs, Gert

2017-01-01

Artificial neural networks (ANNs) trained using backpropagation are powerful learning architectures that have achieved state-of-the-art performance in various benchmarks. Significant effort has been devoted to developing custom silicon devices to accelerate inference in ANNs. Accelerating the training phase, however, has attracted relatively little attention. In this paper, we describe a hardware-efficient on-line learning technique for feedforward multi-layer ANNs that is based on pipelined backpropagation. Learning is performed in parallel with inference in the forward pass, removing the need for an explicit backward pass and requiring no extra weight lookup. By using binary state variables in the feedforward network and ternary errors in truncated-error backpropagation, the need for any multiplications in the forward and backward passes is removed, and memory requirements for the pipelining are drastically reduced. Further reduction in addition operations owing to the sparsity in the forward neural and backpropagating error signal paths contributes to highly efficient hardware implementation. For proof-of-concept validation, we demonstrate on-line learning of MNIST handwritten digit classification on a Spartan 6 FPGA interfacing with an external 1Gb DDR2 DRAM, that shows small degradation in test error performance compared to an equivalently sized binary ANN trained off-line using standard back-propagation and exact errors. Our results highlight an attractive synergy between pipelined backpropagation and binary-state networks in substantially reducing computation and memory requirements, making pipelined on-line learning practical in deep networks. PMID:28932180

Hardware-Efficient On-line Learning through Pipelined Truncated-Error Backpropagation in Binary-State Networks.

PubMed

Mostafa, Hesham; Pedroni, Bruno; Sheik, Sadique; Cauwenberghs, Gert

2017-01-01

Artificial neural networks (ANNs) trained using backpropagation are powerful learning architectures that have achieved state-of-the-art performance in various benchmarks. Significant effort has been devoted to developing custom silicon devices to accelerate inference in ANNs. Accelerating the training phase, however, has attracted relatively little attention. In this paper, we describe a hardware-efficient on-line learning technique for feedforward multi-layer ANNs that is based on pipelined backpropagation. Learning is performed in parallel with inference in the forward pass, removing the need for an explicit backward pass and requiring no extra weight lookup. By using binary state variables in the feedforward network and ternary errors in truncated-error backpropagation, the need for any multiplications in the forward and backward passes is removed, and memory requirements for the pipelining are drastically reduced. Further reduction in addition operations owing to the sparsity in the forward neural and backpropagating error signal paths contributes to highly efficient hardware implementation. For proof-of-concept validation, we demonstrate on-line learning of MNIST handwritten digit classification on a Spartan 6 FPGA interfacing with an external 1Gb DDR2 DRAM, that shows small degradation in test error performance compared to an equivalently sized binary ANN trained off-line using standard back-propagation and exact errors. Our results highlight an attractive synergy between pipelined backpropagation and binary-state networks in substantially reducing computation and memory requirements, making pipelined on-line learning practical in deep networks.

Neural network based feed-forward high density associative memory

NASA Technical Reports Server (NTRS)

Daud, T.; Moopenn, A.; Lamb, J. L.; Ramesham, R.; Thakoor, A. P.

1987-01-01

A novel thin film approach to neural-network-based high-density associative memory is described. The information is stored locally in a memory matrix of passive, nonvolatile, binary connection elements with a potential to achieve a storage density of 10 to the 9th bits/sq cm. Microswitches based on memory switching in thin film hydrogenated amorphous silicon, and alternatively in manganese oxide, have been used as programmable read-only memory elements. Low-energy switching has been ascertained in both these materials. Fabrication and testing of memory matrix is described. High-speed associative recall approaching 10 to the 7th bits/sec and high storage capacity in such a connection matrix memory system is also described.

An "intelligent" approach based on side-by-side cascade-correlation neural networks for estimating thermophysical properties from photothermal responses

NASA Astrophysics Data System (ADS)

Grieu, Stéphane; Faugeroux, Olivier; Traoré, Adama; Claudet, Bernard; Bodnar, Jean-Luc

2015-01-01

In the present paper, an artificial-intelligence-based approach dealing with the estimation of thermophysical properties is designed and evaluated. This new and "intelligent" approach makes use of photothermal responses obtained when subjecting materials to a light flux. So, the main objective of the present work was to estimate simultaneously both the thermal diffusivity and conductivity of materials, from front-face or rear-face photothermal responses to pseudo random binary signals. To this end, we used side-by-side feedforward neural networks trained with the cascade-correlation algorithm. In addition, computation time was a key point to consider. That is why the developed algorithms are computationally tractable.

Development and Validation of a Deep Neural Network Model for Prediction of Postoperative In-hospital Mortality.

PubMed

Lee, Christine K; Hofer, Ira; Gabel, Eilon; Baldi, Pierre; Cannesson, Maxime

2018-04-17

The authors tested the hypothesis that deep neural networks trained on intraoperative features can predict postoperative in-hospital mortality. The data used to train and validate the algorithm consists of 59,985 patients with 87 features extracted at the end of surgery. Feed-forward networks with a logistic output were trained using stochastic gradient descent with momentum. The deep neural networks were trained on 80% of the data, with 20% reserved for testing. The authors assessed improvement of the deep neural network by adding American Society of Anesthesiologists (ASA) Physical Status Classification and robustness of the deep neural network to a reduced feature set. The networks were then compared to ASA Physical Status, logistic regression, and other published clinical scores including the Surgical Apgar, Preoperative Score to Predict Postoperative Mortality, Risk Quantification Index, and the Risk Stratification Index. In-hospital mortality in the training and test sets were 0.81% and 0.73%. The deep neural network with a reduced feature set and ASA Physical Status classification had the highest area under the receiver operating characteristics curve, 0.91 (95% CI, 0.88 to 0.93). The highest logistic regression area under the curve was found with a reduced feature set and ASA Physical Status (0.90, 95% CI, 0.87 to 0.93). The Risk Stratification Index had the highest area under the receiver operating characteristics curve, at 0.97 (95% CI, 0.94 to 0.99). Deep neural networks can predict in-hospital mortality based on automatically extractable intraoperative data, but are not (yet) superior to existing methods.

Feedback enhances feedforward figure-ground segmentation by changing firing mode.

PubMed

Supèr, Hans; Romeo, August

2011-01-01

In the visual cortex, feedback projections are conjectured to be crucial in figure-ground segregation. However, the precise function of feedback herein is unclear. Here we tested a hypothetical model of reentrant feedback. We used a previous developed 2-layered feedforward spiking network that is able to segregate figure from ground and included feedback connections. Our computer model data show that without feedback, neurons respond with regular low-frequency (∼9 Hz) bursting to a figure-ground stimulus. After including feedback the firing pattern changed into a regular (tonic) spiking pattern. In this state, we found an extra enhancement of figure responses and a further suppression of background responses resulting in a stronger figure-ground signal. Such push-pull effect was confirmed by comparing the figure-ground responses with the responses to a homogenous texture. We propose that feedback controls figure-ground segregation by influencing the neural firing patterns of feedforward projecting neurons.

Multisensory visual servoing by a neural network.

PubMed

Wei, G Q; Hirzinger, G

1999-01-01

Conventional computer vision methods for determining a robot's end-effector motion based on sensory data needs sensor calibration (e.g., camera calibration) and sensor-to-hand calibration (e.g., hand-eye calibration). This involves many computations and even some difficulties, especially when different kinds of sensors are involved. In this correspondence, we present a neural network approach to the motion determination problem without any calibration. Two kinds of sensory data, namely, camera images and laser range data, are used as the input to a multilayer feedforward network to associate the direct transformation from the sensory data to the required motions. This provides a practical sensor fusion method. Using a recursive motion strategy and in terms of a network correction, we relax the requirement for the exactness of the learned transformation. Another important feature of our work is that the goal position can be changed without having to do network retraining. Experimental results show the effectiveness of our method.

A neural network approach for the blind deconvolution of turbulent flows

NASA Astrophysics Data System (ADS)

Maulik, R.; San, O.

2017-11-01

We present a single-layer feedforward artificial neural network architecture trained through a supervised learning approach for the deconvolution of flow variables from their coarse grained computations such as those encountered in large eddy simulations. We stress that the deconvolution procedure proposed in this investigation is blind, i.e. the deconvolved field is computed without any pre-existing information about the filtering procedure or kernel. This may be conceptually contrasted to the celebrated approximate deconvolution approaches where a filter shape is predefined for an iterative deconvolution process. We demonstrate that the proposed blind deconvolution network performs exceptionally well in the a-priori testing of both two-dimensional Kraichnan and three-dimensional Kolmogorov turbulence and shows promise in forming the backbone of a physics-augmented data-driven closure for the Navier-Stokes equations.

Modelling the Flow Stress of Alloy 316L using a Multi-Layered Feed Forward Neural Network with Bayesian Regularization

NASA Astrophysics Data System (ADS)

Abiriand Bhekisipho Twala, Olufunminiyi

2017-08-01

In this paper, a multilayer feedforward neural network with Bayesian regularization constitutive model is developed for alloy 316L during high strain rate and high temperature plastic deformation. The input variables are strain rate, temperature and strain while the output value is the flow stress of the material. The results show that the use of Bayesian regularized technique reduces the potential of overfitting and overtraining. The prediction quality of the model is thereby improved. The model predictions are in good agreement with experimental measurements. The measurement data used for the network training and model comparison were taken from relevant literature. The developed model is robust as it can be generalized to deformation conditions slightly below or above the training dataset.

A neural network approach for image reconstruction in electron magnetic resonance tomography.

PubMed

Durairaj, D Christopher; Krishna, Murali C; Murugesan, Ramachandran

2007-10-01

An object-oriented, artificial neural network (ANN) based, application system for reconstruction of two-dimensional spatial images in electron magnetic resonance (EMR) tomography is presented. The standard back propagation algorithm is utilized to train a three-layer sigmoidal feed-forward, supervised, ANN to perform the image reconstruction. The network learns the relationship between the 'ideal' images that are reconstructed using filtered back projection (FBP) technique and the corresponding projection data (sinograms). The input layer of the network is provided with a training set that contains projection data from various phantoms as well as in vivo objects, acquired from an EMR imager. Twenty five different network configurations are investigated to test the ability of the generalization of the network. The trained ANN then reconstructs two-dimensional temporal spatial images that present the distribution of free radicals in biological systems. Image reconstruction by the trained neural network shows better time complexity than the conventional iterative reconstruction algorithms such as multiplicative algebraic reconstruction technique (MART). The network is further explored for image reconstruction from 'noisy' EMR data and the results show better performance than the FBP method. The network is also tested for its ability to reconstruct from limited-angle EMR data set.

Prediction of Air Pollutants Concentration Based on an Extreme Learning Machine: The Case of Hong Kong

PubMed Central

Zhang, Jiangshe; Ding, Weifu

2017-01-01

With the development of the economy and society all over the world, most metropolitan cities are experiencing elevated concentrations of ground-level air pollutants. It is urgent to predict and evaluate the concentration of air pollutants for some local environmental or health agencies. Feed-forward artificial neural networks have been widely used in the prediction of air pollutants concentration. However, there are some drawbacks, such as the low convergence rate and the local minimum. The extreme learning machine for single hidden layer feed-forward neural networks tends to provide good generalization performance at an extremely fast learning speed. The major sources of air pollutants in Hong Kong are mobile, stationary, and from trans-boundary sources. We propose predicting the concentration of air pollutants by the use of trained extreme learning machines based on the data obtained from eight air quality parameters in two monitoring stations, including Sham Shui Po and Tap Mun in Hong Kong for six years. The experimental results show that our proposed algorithm performs better on the Hong Kong data both quantitatively and qualitatively. Particularly, our algorithm shows better predictive ability, with R2 increased and root mean square error values decreased respectively. PMID:28125034

A novel method to produce nonlinear empirical physical formulas for experimental nonlinear electro-optical responses of doped nematic liquid crystals: Feedforward neural network approach

NASA Astrophysics Data System (ADS)

Yildiz, Nihat; San, Sait Eren; Okutan, Mustafa; Kaya, Hüseyin

2010-04-01

Among other significant obstacles, inherent nonlinearity in experimental physical response data poses severe difficulty in empirical physical formula (EPF) construction. In this paper, we applied a novel method (namely layered feedforward neural network (LFNN) approach) to produce explicit nonlinear EPFs for experimental nonlinear electro-optical responses of doped nematic liquid crystals (NLCs). Our motivation was that, as we showed in a previous theoretical work, an appropriate LFNN, due to its exceptional nonlinear function approximation capabilities, is highly relevant to EPF construction. Therefore, in this paper, we obtained excellently produced LFNN approximation functions as our desired EPFs for above-mentioned highly nonlinear response data of NLCs. In other words, by using suitable LFNNs, we successfully fitted the experimentally measured response and predicted the new (yet-to-be measured) response data. The experimental data (response versus input) were diffraction and dielectric properties versus bias voltage; and they were all taken from our previous experimental work. We conclude that in general, LFNN can be applied to construct various types of EPFs for the corresponding various nonlinear physical perturbation (thermal, electronic, molecular, electric, optical, etc.) data of doped NLCs.

Scalable learning method for feedforward neural networks using minimal-enclosing-ball approximation.

PubMed

Wang, Jun; Deng, Zhaohong; Luo, Xiaoqing; Jiang, Yizhang; Wang, Shitong

2016-06-01

Training feedforward neural networks (FNNs) is one of the most critical issues in FNNs studies. However, most FNNs training methods cannot be directly applied for very large datasets because they have high computational and space complexity. In order to tackle this problem, the CCMEB (Center-Constrained Minimum Enclosing Ball) problem in hidden feature space of FNN is discussed and a novel learning algorithm called HFSR-GCVM (hidden-feature-space regression using generalized core vector machine) is developed accordingly. In HFSR-GCVM, a novel learning criterion using L2-norm penalty-based ε-insensitive function is formulated and the parameters in the hidden nodes are generated randomly independent of the training sets. Moreover, the learning of parameters in its output layer is proved equivalent to a special CCMEB problem in FNN hidden feature space. As most CCMEB approximation based machine learning algorithms, the proposed HFSR-GCVM training algorithm has the following merits: The maximal training time of the HFSR-GCVM training is linear with the size of training datasets and the maximal space consumption is independent of the size of training datasets. The experiments on regression tasks confirm the above conclusions. Copyright © 2016 Elsevier Ltd. All rights reserved.

Neural-network-based state feedback control of a nonlinear discrete-time system in nonstrict feedback form.

PubMed

Jagannathan, Sarangapani; He, Pingan

2008-12-01

In this paper, a suite of adaptive neural network (NN) controllers is designed to deliver a desired tracking performance for the control of an unknown, second-order, nonlinear discrete-time system expressed in nonstrict feedback form. In the first approach, two feedforward NNs are employed in the controller with tracking error as the feedback variable whereas in the adaptive critic NN architecture, three feedforward NNs are used. In the adaptive critic architecture, two action NNs produce virtual and actual control inputs, respectively, whereas the third critic NN approximates certain strategic utility function and its output is employed for tuning action NN weights in order to attain the near-optimal control action. Both the NN control methods present a well-defined controller design and the noncausal problem in discrete-time backstepping design is avoided via NN approximation. A comparison between the controller methodologies is highlighted. The stability analysis of the closed-loop control schemes is demonstrated. The NN controller schemes do not require an offline learning phase and the NN weights can be initialized at zero or random. Results show that the performance of the proposed controller schemes is highly satisfactory while meeting the closed-loop stability.

A new approach to applying feedforward neural networks to the prediction of musculoskeletal disorder risk.

PubMed

Chen, C L; Kaber, D B; Dempsey, P G

2000-06-01

A new and improved method to feedforward neural network (FNN) development for application to data classification problems, such as the prediction of levels of low-back disorder (LBD) risk associated with industrial jobs, is presented. Background on FNN development for data classification is provided along with discussions of previous research and neighborhood (local) solution search methods for hard combinatorial problems. An analytical study is presented which compared prediction accuracy of a FNN based on an error-back propagation (EBP) algorithm with the accuracy of a FNN developed by considering results of local solution search (simulated annealing) for classifying industrial jobs as posing low or high risk for LBDs. The comparison demonstrated superior performance of the FNN generated using the new method. The architecture of this FNN included fewer input (predictor) variables and hidden neurons than the FNN developed based on the EBP algorithm. Independent variable selection methods and the phenomenon of 'overfitting' in FNN (and statistical model) generation for data classification are discussed. The results are supportive of the use of the new approach to FNN development for applications to musculoskeletal disorders and risk forecasting in other domains.

Aerodynamic parameters from distributed heterogeneous CNT hair sensors with a feedforward neural network.

PubMed

Magar, Kaman Thapa; Reich, Gregory W; Kondash, Corey; Slinker, Keith; Pankonien, Alexander M; Baur, Jeffery W; Smyers, Brian

2016-11-10

Distributed arrays of artificial hair sensors have bio-like sensing capabilities to obtain spatial and temporal surface flow information which is an important aspect of an effective fly-by-feel system. The spatiotemporal surface flow measurement enables further exploration of additional flow features such as flow stagnation, separation, and reattachment points. Due to their inherent robustness and fault tolerant capability, distributed arrays of hair sensors are well equipped to assess the aerodynamic and flow states in adverse conditions. In this paper, a local flow measurement from an array of artificial hair sensors in a wind tunnel experiment is used with a feedforward artificial neural network to predict aerodynamic parameters such as lift coefficient, moment coefficient, free-stream velocity, and angle of attack on an airfoil. We find the prediction error within 6% and 10% for lift and moment coefficients. The error for free-stream velocity and angle of attack were within 0.12 mph and 0.37 degrees. Knowledge of these parameters are key to finding the real time forces and moments which paves the way for effective control design to increase flight agility, stability, and maneuverability.

Modeling constitutive behavior of a 15Cr-15Ni-2.2Mo-Ti modified austenitic stainless steel under hot compression using artificial neural network

NASA Astrophysics Data System (ADS)

Mandal, Sumantra

2006-11-01

ABSTRACT In this paper, an artificial neural network (ANN) model has been suggested to predict the constitutive flow behavior of a 15Cr-15Ni-2.2Mo-Ti modified austenitic stainless steel under hot deformation. Hot compression tests in the temperature range 850°C- 1250°C and strain rate range 10-3-102 s-1 were carried out. These tests provided the required data for training the neural network and for subsequent testing. The inputs of the neural network are strain, log strain rate and temperature while flow stress is obtained as output. A three layer feed-forward network with ten neurons in a single hidden layer and back-propagation learning algorithm has been employed. A very good correlation between experimental and predicted result has been obtained. The effect of temperature and strain rate on flow behavior has been simulated employing the ANN model. The results have been found to be consistent with the metallurgical trend. Finally, a monte carlo analiysis has been carried out to find out the noise sensitivity of the developed model.

Unorganized machines for seasonal streamflow series forecasting.

PubMed

Siqueira, Hugo; Boccato, Levy; Attux, Romis; Lyra, Christiano

2014-05-01

Modern unorganized machines--extreme learning machines and echo state networks--provide an elegant balance between processing capability and mathematical simplicity, circumventing the difficulties associated with the conventional training approaches of feedforward/recurrent neural networks (FNNs/RNNs). This work performs a detailed investigation of the applicability of unorganized architectures to the problem of seasonal streamflow series forecasting, considering scenarios associated with four Brazilian hydroelectric plants and four distinct prediction horizons. Experimental results indicate the pertinence of these models to the focused task.

Decoding of Human Movements Based on Deep Brain Local Field Potentials Using Ensemble Neural Networks

PubMed Central

2017-01-01

Decoding neural activities related to voluntary and involuntary movements is fundamental to understanding human brain motor circuits and neuromotor disorders and can lead to the development of neuromotor prosthetic devices for neurorehabilitation. This study explores using recorded deep brain local field potentials (LFPs) for robust movement decoding of Parkinson's disease (PD) and Dystonia patients. The LFP data from voluntary movement activities such as left and right hand index finger clicking were recorded from patients who underwent surgeries for implantation of deep brain stimulation electrodes. Movement-related LFP signal features were extracted by computing instantaneous power related to motor response in different neural frequency bands. An innovative neural network ensemble classifier has been proposed and developed for accurate prediction of finger movement and its forthcoming laterality. The ensemble classifier contains three base neural network classifiers, namely, feedforward, radial basis, and probabilistic neural networks. The majority voting rule is used to fuse the decisions of the three base classifiers to generate the final decision of the ensemble classifier. The overall decoding performance reaches a level of agreement (kappa value) at about 0.729 ± 0.16 for decoding movement from the resting state and about 0.671 ± 0.14 for decoding left and right visually cued movements. PMID:29201041

Neural networks with local receptive fields and superlinear VC dimension.

PubMed

Schmitt, Michael

2002-04-01

Local receptive field neurons comprise such well-known and widely used unit types as radial basis function (RBF) neurons and neurons with center-surround receptive field. We study the Vapnik-Chervonenkis (VC) dimension of feedforward neural networks with one hidden layer of these units. For several variants of local receptive field neurons, we show that the VC dimension of these networks is superlinear. In particular, we establish the bound Omega(W log k) for any reasonably sized network with W parameters and k hidden nodes. This bound is shown to hold for discrete center-surround receptive field neurons, which are physiologically relevant models of cells in the mammalian visual system, for neurons computing a difference of gaussians, which are popular in computational vision, and for standard RBF neurons, a major alternative to sigmoidal neurons in artificial neural networks. The result for RBF neural networks is of particular interest since it answers a question that has been open for several years. The results also give rise to lower bounds for networks with fixed input dimension. Regarding constants, all bounds are larger than those known thus far for similar architectures with sigmoidal neurons. The superlinear lower bounds contrast with linear upper bounds for single local receptive field neurons also derived here.

Prediction of Bispectral Index during Target-controlled Infusion of Propofol and Remifentanil: A Deep Learning Approach.

PubMed

Lee, Hyung-Chul; Ryu, Ho-Geol; Chung, Eun-Jin; Jung, Chul-Woo

2018-03-01

The discrepancy between predicted effect-site concentration and measured bispectral index is problematic during intravenous anesthesia with target-controlled infusion of propofol and remifentanil. We hypothesized that bispectral index during total intravenous anesthesia would be more accurately predicted by a deep learning approach. Long short-term memory and the feed-forward neural network were sequenced to simulate the pharmacokinetic and pharmacodynamic parts of an empirical model, respectively, to predict intraoperative bispectral index during combined use of propofol and remifentanil. Inputs of long short-term memory were infusion histories of propofol and remifentanil, which were retrieved from target-controlled infusion pumps for 1,800 s at 10-s intervals. Inputs of the feed-forward network were the outputs of long short-term memory and demographic data such as age, sex, weight, and height. The final output of the feed-forward network was the bispectral index. The performance of bispectral index prediction was compared between the deep learning model and previously reported response surface model. The model hyperparameters comprised 8 memory cells in the long short-term memory layer and 16 nodes in the hidden layer of the feed-forward network. The model training and testing were performed with separate data sets of 131 and 100 cases. The concordance correlation coefficient (95% CI) were 0.561 (0.560 to 0.562) in the deep learning model, which was significantly larger than that in the response surface model (0.265 [0.263 to 0.266], P < 0.001). The deep learning model-predicted bispectral index during target-controlled infusion of propofol and remifentanil more accurately compared to the traditional model. The deep learning approach in anesthetic pharmacology seems promising because of its excellent performance and extensibility.

Neural Network Optimization of Ligament Stiffnesses for the Enhanced Predictive Ability of a Patient-Specific, Computational Foot/Ankle Model.

PubMed

Chande, Ruchi D; Wayne, Jennifer S

2017-09-01

Computational models of diarthrodial joints serve to inform the biomechanical function of these structures, and as such, must be supplied appropriate inputs for performance that is representative of actual joint function. Inputs for these models are sourced from both imaging modalities as well as literature. The latter is often the source of mechanical properties for soft tissues, like ligament stiffnesses; however, such data are not always available for all the soft tissues nor is it known for patient-specific work. In the current research, a method to improve the ligament stiffness definition for a computational foot/ankle model was sought with the greater goal of improving the predictive ability of the computational model. Specifically, the stiffness values were optimized using artificial neural networks (ANNs); both feedforward and radial basis function networks (RBFNs) were considered. Optimal networks of each type were determined and subsequently used to predict stiffnesses for the foot/ankle model. Ultimately, the predicted stiffnesses were considered reasonable and resulted in enhanced performance of the computational model, suggesting that artificial neural networks can be used to optimize stiffness inputs.

Classification of multispectral image data by the Binary Diamond neural network and by nonparametric, pixel-by-pixel methods

NASA Technical Reports Server (NTRS)

Salu, Yehuda; Tilton, James

1993-01-01

The classification of multispectral image data obtained from satellites has become an important tool for generating ground cover maps. This study deals with the application of nonparametric pixel-by-pixel classification methods in the classification of pixels, based on their multispectral data. A new neural network, the Binary Diamond, is introduced, and its performance is compared with a nearest neighbor algorithm and a back-propagation network. The Binary Diamond is a multilayer, feed-forward neural network, which learns from examples in unsupervised, 'one-shot' mode. It recruits its neurons according to the actual training set, as it learns. The comparisons of the algorithms were done by using a realistic data base, consisting of approximately 90,000 Landsat 4 Thematic Mapper pixels. The Binary Diamond and the nearest neighbor performances were close, with some advantages to the Binary Diamond. The performance of the back-propagation network lagged behind. An efficient nearest neighbor algorithm, the binned nearest neighbor, is described. Ways for improving the performances, such as merging categories, and analyzing nonboundary pixels, are addressed and evaluated.

Neural network simulation of the atmospheric point spread function for the adjacency effect research

NASA Astrophysics Data System (ADS)

Ma, Xiaoshan; Wang, Haidong; Li, Ligang; Yang, Zhen; Meng, Xin

2016-10-01

Adjacency effect could be regarded as the convolution of the atmospheric point spread function (PSF) and the surface leaving radiance. Monte Carlo is a common method to simulate the atmospheric PSF. But it can't obtain analytic expression and the meaningful results can be only acquired by statistical analysis of millions of data. A backward Monte Carlo algorithm was employed to simulate photon emitting and propagating in the atmosphere under different conditions. The PSF was determined by recording the photon-receiving numbers in fixed bin at different position. A multilayer feed-forward neural network with a single hidden layer was designed to learn the relationship between the PSF's and the input condition parameters. The neural network used the back-propagation learning rule for training. Its input parameters involved atmosphere condition, spectrum range, observing geometry. The outputs of the network were photon-receiving numbers in the corresponding bin. Because the output units were too many to be allowed by neural network, the large network was divided into a collection of smaller ones. These small networks could be ran simultaneously on many workstations and/or PCs to speed up the training. It is important to note that the simulated PSF's by Monte Carlo technique in non-nadir viewing angles are more complicated than that in nadir conditions which brings difficulties in the design of the neural network. The results obtained show that the neural network approach could be very useful to compute the atmospheric PSF based on the simulated data generated by Monte Carlo method.

Neural network evaluation of tokamak current profiles for real time control

NASA Astrophysics Data System (ADS)

Wróblewski, Dariusz

1997-02-01

Active feedback control of the current profile, requiring real-time determination of the current profile parameters, is envisioned for tokamaks operating in enhanced confinement regimes. The distribution of toroidal current in a tokamak is now routinely evaluated based on external (magnetic probes, flux loops) and internal (motional Stark effect) measurements of the poloidal magnetic field. However, the analysis involves reconstruction of magnetohydrodynamic equilibrium and is too intensive computationally to be performed in real time. In the present study, a neural network is used to provide a mapping from the magnetic measurements (internal and external) to selected parameters of the safety factor profile. The single-pass, feedforward calculation of output of a trained neural network is very fast, making this approach particularly suitable for real-time applications. The network was trained on a large set of simulated equilibrium data for the DIII-D tokamak. The database encompasses a large variety of current profiles including the hollow current profiles important for reversed central shear operation. The parameters of safety factor profile (a quantity related to the current profile through the magnetic field tilt angle) estimated by the neural network include central safety factor, q0, minimum value of q, qmin, and the location of qmin. Very good performance of the trained neural network both for simulated test data and for experimental datais demonstrated.

Neural network evaluation of tokamak current profiles for real time control (abstract)

NASA Astrophysics Data System (ADS)

Wróblewski, Dariusz

1997-01-01

Active feedback control of the current profile, requiring real-time determination of the current profile parameters, is envisioned for tokamaks operating in enhanced confinement regimes. The distribution of toroidal current in a tokamak is now routinely evaluated based on external (magnetic probes, flux loops) and internal (motional Stark effect) measurements of the poloidal magnetic field. However, the analysis involves reconstruction of magnetohydrodynamic equilibrium and is too intensive computationally to be performed in real time. In the present study, a neural network is used to provide a mapping from the magnetic measurements (internal and external) to selected parameters of the safety factor profile. The single-pass, feedforward calculation of output of a trained neural network is very fast, making this approach particularly suitable for real-time applications. The network was trained on a large set of simulated equilibrium data for the DIII-D tokamak. The database encompasses a large variety of current profiles including the hollow current profiles important for reversed central shear operation. The parameters of safety factor profile (a quantity related to the current profile through the magnetic field tilt angle) estimated by the neural network include central safety factor, q0, minimum value of q, qmin, and the location of qmin. Very good performance of the trained neural network both for simulated test data and for experimental data is demonstrated.

Self-organizing linear output map (SOLO): An artificial neural network suitable for hydrologic modeling and analysis

NASA Astrophysics Data System (ADS)

Hsu, Kuo-Lin; Gupta, Hoshin V.; Gao, Xiaogang; Sorooshian, Soroosh; Imam, Bisher

2002-12-01

Artificial neural networks (ANNs) can be useful in the prediction of hydrologic variables, such as streamflow, particularly when the underlying processes have complex nonlinear interrelationships. However, conventional ANN structures suffer from network training issues that significantly limit their widespread application. This paper presents a multivariate ANN procedure entitled self-organizing linear output map (SOLO), whose structure has been designed for rapid, precise, and inexpensive estimation of network structure/parameters and system outputs. More important, SOLO provides features that facilitate insight into the underlying processes, thereby extending its usefulness beyond forecast applications as a tool for scientific investigations. These characteristics are demonstrated using a classic rainfall-runoff forecasting problem. Various aspects of model performance are evaluated in comparison with other commonly used modeling approaches, including multilayer feedforward ANNs, linear time series modeling, and conceptual rainfall-runoff modeling.

New recursive-least-squares algorithms for nonlinear active control of sound and vibration using neural networks.

PubMed

Bouchard, M

2001-01-01

In recent years, a few articles describing the use of neural networks for nonlinear active control of sound and vibration were published. Using a control structure with two multilayer feedforward neural networks (one as a nonlinear controller and one as a nonlinear plant model), steepest descent algorithms based on two distinct gradient approaches were introduced for the training of the controller network. The two gradient approaches were sometimes called the filtered-x approach and the adjoint approach. Some recursive-least-squares algorithms were also introduced, using the adjoint approach. In this paper, an heuristic procedure is introduced for the development of recursive-least-squares algorithms based on the filtered-x and the adjoint gradient approaches. This leads to the development of new recursive-least-squares algorithms for the training of the controller neural network in the two networks structure. These new algorithms produce a better convergence performance than previously published algorithms. Differences in the performance of algorithms using the filtered-x and the adjoint gradient approaches are discussed in the paper. The computational load of the algorithms discussed in the paper is evaluated for multichannel systems of nonlinear active control. Simulation results are presented to compare the convergence performance of the algorithms, showing the convergence gain provided by the new algorithms.

Reciprocal inhibition of inhibition: A circuit motif for flexible categorization in stimulus selection

PubMed Central

Knudsen, Eric I.

2011-01-01

As a precursor to the selection of a stimulus for gaze and attention, a midbrain network categorizes stimuli into “strongest” and “others.” The categorization tracks flexibly, in real-time, the absolute strength of the strongest stimulus. In this study, we take a first principles approach to computations that are essential for such categorization. We demonstrate that classical feedforward lateral inhibition cannot produce flexible categorization. However, circuits in which the strength of lateral inhibition varies with the relative strength of competing stimuli categorize successfully. One particular implementation - reciprocal inhibition of feedforward lateral inhibition – is structurally the simplest, and it outperforms others in flexibly categorizing rapidly and reliably. Strong predictions of this anatomically supported circuit model are validated by neural responses measured in the owl midbrain. The results demonstrate the extraordinary power of a remarkably simple, neurally grounded circuit motif in producing flexible categorization, a computation fundamental to attention, perception, and decision-making. PMID:22243757

Improvement of single wavelength-based Thai jasmine rice identification with elliptic Fourier descriptor and neural network analysis

NASA Astrophysics Data System (ADS)

Suwansukho, Kajpanya; Sumriddetchkajorn, Sarun; Buranasiri, Prathan

2012-11-01

Instead of considering only the amount of fluorescent signal spatially distributed on the image of milled rice grains this paper shows how our single-wavelength spectral-imaging-based Thai jasmine (KDML105) rice identification system can be improved by analyzing the shape and size of the image of each milled rice variety especially during the image threshold operation. The image of each milled rice variety is expressed as chain codes and elliptic Fourier coefficients. After that, a feed-forward back-propagation neural network model is applied, resulting in an improved average FAR of 11.0% and FRR of 19.0% in identifying KDML105 milled rice from the unwanted four milled rice varieties.

Backpropagation and ordered derivatives in the time scales calculus.

PubMed

Seiffertt, John; Wunsch, Donald C

2010-08-01

Backpropagation is the most widely used neural network learning technique. It is based on the mathematical notion of an ordered derivative. In this paper, we present a formulation of ordered derivatives and the backpropagation training algorithm using the important emerging area of mathematics known as the time scales calculus. This calculus, with its potential for application to a wide variety of inter-disciplinary problems, is becoming a key area of mathematics. It is capable of unifying continuous and discrete analysis within one coherent theoretical framework. Using this calculus, we present here a generalization of backpropagation which is appropriate for cases beyond the specifically continuous or discrete. We develop a new multivariate chain rule of this calculus, define ordered derivatives on time scales, prove a key theorem about them, and derive the backpropagation weight update equations for a feedforward multilayer neural network architecture. By drawing together the time scales calculus and the area of neural network learning, we present the first connection of two major fields of research.

Cellular automata simulation of topological effects on the dynamics of feed-forward motifs

PubMed Central

Apte, Advait A; Cain, John W; Bonchev, Danail G; Fong, Stephen S

2008-01-01

Background Feed-forward motifs are important functional modules in biological and other complex networks. The functionality of feed-forward motifs and other network motifs is largely dictated by the connectivity of the individual network components. While studies on the dynamics of motifs and networks are usually devoted to the temporal or spatial description of processes, this study focuses on the relationship between the specific architecture and the overall rate of the processes of the feed-forward family of motifs, including double and triple feed-forward loops. The search for the most efficient network architecture could be of particular interest for regulatory or signaling pathways in biology, as well as in computational and communication systems. Results Feed-forward motif dynamics were studied using cellular automata and compared with differential equation modeling. The number of cellular automata iterations needed for a 100% conversion of a substrate into a target product was used as an inverse measure of the transformation rate. Several basic topological patterns were identified that order the specific feed-forward constructions according to the rate of dynamics they enable. At the same number of network nodes and constant other parameters, the bi-parallel and tri-parallel motifs provide higher network efficacy than single feed-forward motifs. Additionally, a topological property of isodynamicity was identified for feed-forward motifs where different network architectures resulted in the same overall rate of the target production. Conclusion It was shown for classes of structural motifs with feed-forward architecture that network topology affects the overall rate of a process in a quantitatively predictable manner. These fundamental results can be used as a basis for simulating larger networks as combinations of smaller network modules with implications on studying synthetic gene circuits, small regulatory systems, and eventually dynamic whole-cell models. PMID:18304325

Retrieving Tract Variables From Acoustics: A Comparison of Different Machine Learning Strategies.

PubMed

Mitra, Vikramjit; Nam, Hosung; Espy-Wilson, Carol Y; Saltzman, Elliot; Goldstein, Louis

2010-09-13

Many different studies have claimed that articulatory information can be used to improve the performance of automatic speech recognition systems. Unfortunately, such articulatory information is not readily available in typical speaker-listener situations. Consequently, such information has to be estimated from the acoustic signal in a process which is usually termed "speech-inversion." This study aims to propose and compare various machine learning strategies for speech inversion: Trajectory mixture density networks (TMDNs), feedforward artificial neural networks (FF-ANN), support vector regression (SVR), autoregressive artificial neural network (AR-ANN), and distal supervised learning (DSL). Further, using a database generated by the Haskins Laboratories speech production model, we test the claim that information regarding constrictions produced by the distinct organs of the vocal tract (vocal tract variables) is superior to flesh-point information (articulatory pellet trajectories) for the inversion process.

Objective assessment of MPEG-2 video quality

NASA Astrophysics Data System (ADS)

Gastaldo, Paolo; Zunino, Rodolfo; Rovetta, Stefano

2002-07-01

The increasing use of video compression standards in broadcasting television systems has required, in recent years, the development of video quality measurements that take into account artifacts specifically caused by digital compression techniques. In this paper we present a methodology for the objective quality assessment of MPEG video streams by using circular back-propagation feedforward neural networks. Mapping neural networks can render nonlinear relationships between objective features and subjective judgments, thus avoiding any simplifying assumption on the complexity of the model. The neural network processes an instantaneous set of input values, and yields an associated estimate of perceived quality. Therefore, the neural-network approach turns objective quality assessment into adaptive modeling of subjective perception. The objective features used for the estimate are chosen according to the assessed relevance to perceived quality and are continuously extracted in real time from compressed video streams. The overall system mimics perception but does not require any analytical model of the underlying physical phenomenon. The capability to process compressed video streams represents an important advantage over existing approaches, like avoiding the stream-decoding process greatly enhances real-time performance. Experimental results confirm that the system provides satisfactory, continuous-time approximations for actual scoring curves concerning real test videos.

Neural network application to aircraft control system design

NASA Technical Reports Server (NTRS)

Troudet, Terry; Garg, Sanjay; Merrill, Walter C.

1991-01-01

The feasibility of using artificial neural networks as control systems for modern, complex aerospace vehicles is investigated via an example aircraft control design study. The problem considered is that of designing a controller for an integrated airframe/propulsion longitudinal dynamics model of a modern fighter aircraft to provide independent control of pitch rate and airspeed responses to pilot command inputs. An explicit model following controller using H infinity control design techniques is first designed to gain insight into the control problem as well as to provide a baseline for evaluation of the neurocontroller. Using the model of the desired dynamics as a command generator, a multilayer feedforward neural network is trained to control the vehicle model within the physical limitations of the actuator dynamics. This is achieved by minimizing an objective function which is a weighted sum of tracking errors and control input commands and rates. To gain insight in the neurocontrol, linearized representations of the nonlinear neurocontroller are analyzed along a commanded trajectory. Linear robustness analysis tools are then applied to the linearized neurocontroller models and to the baseline H infinity based controller. Future areas of research are identified to enhance the practical applicability of neural networks to flight control design.

Neural network application to aircraft control system design

NASA Technical Reports Server (NTRS)

Troudet, Terry; Garg, Sanjay; Merrill, Walter C.

1991-01-01

The feasibility of using artificial neural network as control systems for modern, complex aerospace vehicles is investigated via an example aircraft control design study. The problem considered is that of designing a controller for an integrated airframe/propulsion longitudinal dynamics model of a modern fighter aircraft to provide independent control of pitch rate and airspeed responses to pilot command inputs. An explicit model following controller using H infinity control design techniques is first designed to gain insight into the control problem as well as to provide a baseline for evaluation of the neurocontroller. Using the model of the desired dynamics as a command generator, a multilayer feedforward neural network is trained to control the vehicle model within the physical limitations of the actuator dynamics. This is achieved by minimizing an objective function which is a weighted sum of tracking errors and control input commands and rates. To gain insight in the neurocontrol, linearized representations of the nonlinear neurocontroller are analyzed along a commanded trajectory. Linear robustness analysis tools are then applied to the linearized neurocontroller models and to the baseline H infinity based controller. Future areas of research identified to enhance the practical applicability of neural networks to flight control design.

Overreliance on auditory feedback may lead to sound/syllable repetitions: simulations of stuttering and fluency-inducing conditions with a neural model of speech production

PubMed Central

Civier, Oren; Tasko, Stephen M.; Guenther, Frank H.

2010-01-01

This paper investigates the hypothesis that stuttering may result in part from impaired readout of feedforward control of speech, which forces persons who stutter (PWS) to produce speech with a motor strategy that is weighted too much toward auditory feedback control. Over-reliance on feedback control leads to production errors which, if they grow large enough, can cause the motor system to “reset” and repeat the current syllable. This hypothesis is investigated using computer simulations of a “neurally impaired” version of the DIVA model, a neural network model of speech acquisition and production. The model’s outputs are compared to published acoustic data from PWS’ fluent speech, and to combined acoustic and articulatory movement data collected from the dysfluent speech of one PWS. The simulations mimic the errors observed in the PWS subject’s speech, as well as the repairs of these errors. Additional simulations were able to account for enhancements of fluency gained by slowed/prolonged speech and masking noise. Together these results support the hypothesis that many dysfluencies in stuttering are due to a bias away from feedforward control and toward feedback control. PMID:20831971

Neural Network Control of a Magnetically Suspended Rotor System

NASA Technical Reports Server (NTRS)

Choi, Benjamin B.

1998-01-01

Magnetic bearings offer significant advantages because they do not come into contact with other parts during operation, which can reduce maintenance. Higher speeds, no friction, no lubrication, weight reduction, precise position control, and active damping make them far superior to conventional contact bearings. However, there are technical barriers that limit the application of this technology in industry. One of them is the need for a nonlinear controller that can overcome the system nonlinearity and uncertainty inherent in magnetic bearings. At the NASA Lewis Research Center, a neural network was selected as a nonlinear controller because it generates a neural model without any detailed information regarding the internal working of the magnetic bearing system. It can be used even for systems that are too complex for an accurate system model to be derived. A feed-forward architecture with a back-propagation learning algorithm was selected because of its proven performance, accuracy, and relatively easy implementation.

The mechanics of state dependent neural correlations

PubMed Central

Doiron, Brent; Litwin-Kumar, Ashok; Rosenbaum, Robert; Ocker, Gabriel K.; Josić, Krešimir

2016-01-01

Simultaneous recordings from large neural populations are becoming increasingly common. An important feature of the population activity are the trial-to-trial correlated fluctuations of the spike train outputs of recorded neuron pairs. Like the firing rate of single neurons, correlated activity can be modulated by a number of factors, from changes in arousal and attentional state to learning and task engagement. However, the network mechanisms that underlie these changes are not fully understood. We review recent theoretical results that identify three separate biophysical mechanisms that modulate spike train correlations: changes in input correlations, internal fluctuations, and the transfer function of single neurons. We first examine these mechanisms in feedforward pathways, and then show how the same approach can explain the modulation of correlations in recurrent networks. Such mechanistic constraints on the modulation of population activity will be important in statistical analyses of high dimensional neural data. PMID:26906505

Supervised Learning Based on Temporal Coding in Spiking Neural Networks.

PubMed

Mostafa, Hesham

2017-08-01

Gradient descent training techniques are remarkably successful in training analog-valued artificial neural networks (ANNs). Such training techniques, however, do not transfer easily to spiking networks due to the spike generation hard nonlinearity and the discrete nature of spike communication. We show that in a feedforward spiking network that uses a temporal coding scheme where information is encoded in spike times instead of spike rates, the network input-output relation is differentiable almost everywhere. Moreover, this relation is piecewise linear after a transformation of variables. Methods for training ANNs thus carry directly to the training of such spiking networks as we show when training on the permutation invariant MNIST task. In contrast to rate-based spiking networks that are often used to approximate the behavior of ANNs, the networks we present spike much more sparsely and their behavior cannot be directly approximated by conventional ANNs. Our results highlight a new approach for controlling the behavior of spiking networks with realistic temporal dynamics, opening up the potential for using these networks to process spike patterns with complex temporal information.

A continually online-trained neural network controller for brushless DC motor drives

DOE Office of Scientific and Technical Information (OSTI.GOV)

Rubaai, A.; Kotaru, R.; Kankam, M.D.

2000-04-01

In this paper, a high-performance controller with simultaneous online identification and control is designed for brushless dc motor drives. The dynamics of the motor/load are modeled online, and controlled using two different neural network based identification and control schemes, as the system is in operation. In the first scheme, an attempt is made to control the rotor angular speed, utilizing a single three-hidden-layer network. The second scheme attempts to control the stator currents, using a predetermined control law as a function of the estimated states. This schemes incorporates three multilayered feedforward neural networks that are online trained, using the Levenburg-Marquadtmore » training algorithm. The control of the direct and quadrature components of the stator current successfully tracked a wide variety of trajectories after relatively short online training periods. The control strategy adapts to the uncertainties of the motor/load dynamics and, in addition, learns their inherent nonlinearities. Simulation results illustrated that a neurocontroller used in conjunction with adaptive control schemes can result in a flexible control device which may be utilized in a wide range of environments.« less

Combined contributions of feedforward and feedback inputs to bottom-up attention

PubMed Central

Khorsand, Peyman; Moore, Tirin; Soltani, Alireza

2015-01-01

In order to deal with a large amount of information carried by visual inputs entering the brain at any given point in time, the brain swiftly uses the same inputs to enhance processing in one part of visual field at the expense of the others. These processes, collectively called bottom-up attentional selection, are assumed to solely rely on feedforward processing of the external inputs, as it is implied by the nomenclature. Nevertheless, evidence from recent experimental and modeling studies points to the role of feedback in bottom-up attention. Here, we review behavioral and neural evidence that feedback inputs are important for the formation of signals that could guide attentional selection based on exogenous inputs. Moreover, we review results from a modeling study elucidating mechanisms underlying the emergence of these signals in successive layers of neural populations and how they depend on feedback from higher visual areas. We use these results to interpret and discuss more recent findings that can further unravel feedforward and feedback neural mechanisms underlying bottom-up attention. We argue that while it is descriptively useful to separate feedforward and feedback processes underlying bottom-up attention, these processes cannot be mechanistically separated into two successive stages as they occur at almost the same time and affect neural activity within the same brain areas using similar neural mechanisms. Therefore, understanding the interaction and integration of feedforward and feedback inputs is crucial for better understanding of bottom-up attention. PMID:25784883

Application of Multilayer Feedforward Neural Networks to Precipitation Cell-Top Altitude Estimation

NASA Technical Reports Server (NTRS)

Spina, Michelle S.; Schwartz, Michael J.; Staelin, David H.; Gasiewski, Albin J.

1998-01-01

The use of passive 118-GHz O2 observations of rain cells for precipitation cell-top altitude estimation is demonstrated by using a multilayer feed forward neural network retrieval system. Rain cell observations at 118 GHz were compared with estimates of the cell-top altitude obtained by optical stereoscopy. The observations were made with 2 4 km horizontal spatial resolution by using the Millimeter-wave Temperature Sounder (MTS) scanning spectrometer aboard the NASA ER-2 research aircraft during the Genesis of Atlantic Lows Experiment (GALE) and the COoperative Huntsville Meteorological EXperiment (COHMEX) in 1986. The neural network estimator applied to MTS spectral differences between clouds, and nearby clear air yielded an rms discrepancy of 1.76 km for a combined cumulus, mature, and dissipating cell set and 1.44 km for the cumulus-only set. An improvement in rms discrepancy to 1.36 km was achieved by including additional MTS information on the absolute atmospheric temperature profile. An incremental method for training neural networks was developed that yielded robust results, despite the use of as few as 56 training spectra. Comparison of these results with a nonlinear statistical estimator shows that superior results can be obtained with a neural network retrieval system. Imagery of estimated cell-top altitudes was created from 118-GHz spectral imagery gathered from CAMEX, September through October 1993, and from cyclone Oliver, February 7, 1993.

Autoshaped choice in artificial neural networks: implications for behavioral economics and neuroeconomics.

PubMed

Burgos, José E; García-Leal, Óscar

2015-05-01

An existing neural network model of conditioning was used to simulate autoshaped choice. In this phenomenon, pigeons first receive an autoshaping procedure with two keylight stimuli X and Y separately paired with food in a forward-delay manner, intermittently for X and continuously for Y. Then pigeons receive unreinforced choice test trials of X and Y concurrently present. Most pigeons choose Y. This preference for a more valuable response alternative is a form of economic behavior that makes the phenomenon relevant to behavioral economics. The phenomenon also suggests a role for Pavlovian contingencies in economic behavior. The model used, in contrast to others, predicts autoshaping and automaintenance, so it is uniquely positioned to predict autoshaped choice. The model also contemplates neural substrates of economic behavior in neuroeconomics, such as dopaminergic and hippocampal systems. A feedforward neural network architecture was designed to simulate a neuroanatomical differentiation between two environment-behavior relations X-R1 and Y-R2, [corrected] where R1 and R2 denote two different emitted responses (not unconditionally elicited by the reward). Networks with this architecture received a training protocol that simulated an autoshaped-choice procedure. Most networks simulated the phenomenon. Implications for behavioral economics and neuroeconomics, limitations, and the issue of model appraisal are discussed. Copyright © 2015 Elsevier B.V. All rights reserved.

Fuzzy logic and neural networks in artificial intelligence and pattern recognition

NASA Astrophysics Data System (ADS)

Sanchez, Elie

1991-10-01

With the use of fuzzy logic techniques, neural computing can be integrated in symbolic reasoning to solve complex real world problems. In fact, artificial neural networks, expert systems, and fuzzy logic systems, in the context of approximate reasoning, share common features and techniques. A model of Fuzzy Connectionist Expert System is introduced, in which an artificial neural network is designed to construct the knowledge base of an expert system from, training examples (this model can also be used for specifications of rules in fuzzy logic control). Two types of weights are associated with the synaptic connections in an AND-OR structure: primary linguistic weights, interpreted as labels of fuzzy sets, and secondary numerical weights. Cell activation is computed through min-max fuzzy equations of the weights. Learning consists in finding the (numerical) weights and the network topology. This feedforward network is described and first illustrated in a biomedical application (medical diagnosis assistance from inflammatory-syndromes/proteins profiles). Then, it is shown how this methodology can be utilized for handwritten pattern recognition (characters play the role of diagnoses): in a fuzzy neuron describing a number for example, the linguistic weights represent fuzzy sets on cross-detecting lines and the numerical weights reflect the importance (or weakness) of connections between cross-detecting lines and characters.

A Viola-Jones based hybrid face detection framework

NASA Astrophysics Data System (ADS)

Murphy, Thomas M.; Broussard, Randy; Schultz, Robert; Rakvic, Ryan; Ngo, Hau

2013-12-01

Improvements in face detection performance would benefit many applications. The OpenCV library implements a standard solution, the Viola-Jones detector, with a statistically boosted rejection cascade of binary classifiers. Empirical evidence has shown that Viola-Jones underdetects in some instances. This research shows that a truncated cascade augmented by a neural network could recover these undetected faces. A hybrid framework is constructed, with a truncated Viola-Jones cascade followed by an artificial neural network, used to refine the face decision. Optimally, a truncation stage that captured all faces and allowed the neural network to remove the false alarms is selected. A feedforward backpropagation network with one hidden layer is trained to discriminate faces based upon the thresholding (detection) values of intermediate stages of the full rejection cascade. A clustering algorithm is used as a precursor to the neural network, to group significant overlappings. Evaluated on the CMU/VASC Image Database, comparison with an unmodified OpenCV approach shows: (1) a 37% increase in detection rates if constrained by the requirement of no increase in false alarms, (2) a 48% increase in detection rates if some additional false alarms are tolerated, and (3) an 82% reduction in false alarms with no reduction in detection rates. These results demonstrate improved face detection and could address the need for such improvement in various applications.

Neural network for image compression

NASA Astrophysics Data System (ADS)

Panchanathan, Sethuraman; Yeap, Tet H.; Pilache, B.

1992-09-01

In this paper, we propose a new scheme for image compression using neural networks. Image data compression deals with minimization of the amount of data required to represent an image while maintaining an acceptable quality. Several image compression techniques have been developed in recent years. We note that the coding performance of these techniques may be improved by employing adaptivity. Over the last few years neural network has emerged as an effective tool for solving a wide range of problems involving adaptivity and learning. A multilayer feed-forward neural network trained using the backward error propagation algorithm is used in many applications. However, this model is not suitable for image compression because of its poor coding performance. Recently, a self-organizing feature map (SOFM) algorithm has been proposed which yields a good coding performance. However, this algorithm requires a long training time because the network starts with random initial weights. In this paper we have used the backward error propagation algorithm (BEP) to quickly obtain the initial weights which are then used to speedup the training time required by the SOFM algorithm. The proposed approach (BEP-SOFM) combines the advantages of the two techniques and, hence, achieves a good coding performance in a shorter training time. Our simulation results demonstrate the potential gains using the proposed technique.

Correcting wave predictions with artificial neural networks

NASA Astrophysics Data System (ADS)

Makarynskyy, O.; Makarynska, D.

2003-04-01

The predictions of wind waves with different lead times are necessary in a large scope of coastal and open ocean activities. Numerical wave models, which usually provide this information, are based on deterministic equations that do not entirely account for the complexity and uncertainty of the wave generation and dissipation processes. An attempt to improve wave parameters short-term forecasts based on artificial neural networks is reported. In recent years, artificial neural networks have been used in a number of coastal engineering applications due to their ability to approximate the nonlinear mathematical behavior without a priori knowledge of interrelations among the elements within a system. The common multilayer feed-forward networks, with a nonlinear transfer functions in the hidden layers, were developed and employed to forecast the wave characteristics over one hour intervals starting from one up to 24 hours, and to correct these predictions. Three non-overlapping data sets of wave characteristics, both from a buoy, moored roughly 60 miles west of the Aran Islands, west coast of Ireland, were used to train and validate the neural nets involved. The networks were trained with error back propagation algorithm. Time series plots and scatterplots of the wave characteristics as well as tables with statistics show an improvement of the results achieved due to the correction procedure employed.

Two papers on feed-forward networks

NASA Technical Reports Server (NTRS)

Buntine, Wray L.; Weigend, Andreas S.

1991-01-01

Connectionist feed-forward networks, trained with back-propagation, can be used both for nonlinear regression and for (discrete one-of-C) classification, depending on the form of training. This report contains two papers on feed-forward networks. The papers can be read independently. They are intended for the theoretically-aware practitioner or algorithm-designer; however, they also contain a review and comparison of several learning theories so they provide a perspective for the theoretician. The first paper works through Bayesian methods to complement back-propagation in the training of feed-forward networks. The second paper addresses a problem raised by the first: how to efficiently calculate second derivatives on feed-forward networks.

A fuzzy neural network for intelligent data processing

NASA Astrophysics Data System (ADS)

Xie, Wei; Chu, Feng; Wang, Lipo; Lim, Eng Thiam

2005-03-01

In this paper, we describe an incrementally generated fuzzy neural network (FNN) for intelligent data processing. This FNN combines the features of initial fuzzy model self-generation, fast input selection, partition validation, parameter optimization and rule-base simplification. A small FNN is created from scratch -- there is no need to specify the initial network architecture, initial membership functions, or initial weights. Fuzzy IF-THEN rules are constantly combined and pruned to minimize the size of the network while maintaining accuracy; irrelevant inputs are detected and deleted, and membership functions and network weights are trained with a gradient descent algorithm, i.e., error backpropagation. Experimental studies on synthesized data sets demonstrate that the proposed Fuzzy Neural Network is able to achieve accuracy comparable to or higher than both a feedforward crisp neural network, i.e., NeuroRule, and a decision tree, i.e., C4.5, with more compact rule bases for most of the data sets used in our experiments. The FNN has achieved outstanding results for cancer classification based on microarray data. The excellent classification result for Small Round Blue Cell Tumors (SRBCTs) data set is shown. Compared with other published methods, we have used a much fewer number of genes for perfect classification, which will help researchers directly focus their attention on some specific genes and may lead to discovery of deep reasons of the development of cancers and discovery of drugs.

Analyzing Divisia Rules Extracted from a Feedforward Neural Network

DTIC Science & Technology

2006-03-01

assumptions. (Barnett and work, Data Mining, Rule Generation Serletis give a detailed treatment of the the- ory of monetary aggregation [1].) However, 1... Serletis , A. (Eds.) (2000), The The- Swizerland, 1995. ory of Monetary Aggregation, North-H ollandeAmsterdam, Chgaptero , pp.- [11] Vincent A. Schmidt and...gas, Nevada, 2002. sets. Macroeconomic Dynamics, 1:485-512, 1997. Reprinted in Barnett, WA. [12] Vincent A. Schmidt and Jane M. Binner. and Serletis

On the sample complexity of learning for networks of spiking neurons with nonlinear synaptic interactions.

PubMed

Schmitt, Michael

2004-09-01

We study networks of spiking neurons that use the timing of pulses to encode information. Nonlinear interactions model the spatial groupings of synapses on the neural dendrites and describe the computations performed at local branches. Within a theoretical framework of learning we analyze the question of how many training examples these networks must receive to be able to generalize well. Bounds for this sample complexity of learning can be obtained in terms of a combinatorial parameter known as the pseudodimension. This dimension characterizes the computational richness of a neural network and is given in terms of the number of network parameters. Two types of feedforward architectures are considered: constant-depth networks and networks of unconstrained depth. We derive asymptotically tight bounds for each of these network types. Constant depth networks are shown to have an almost linear pseudodimension, whereas the pseudodimension of general networks is quadratic. Networks of spiking neurons that use temporal coding are becoming increasingly more important in practical tasks such as computer vision, speech recognition, and motor control. The question of how well these networks generalize from a given set of training examples is a central issue for their successful application as adaptive systems. The results show that, although coding and computation in these networks is quite different and in many cases more powerful, their generalization capabilities are at least as good as those of traditional neural network models.

Neural-network-based state of health diagnostics for an automated radioxenon sampler/analyzer

NASA Astrophysics Data System (ADS)

Keller, Paul E.; Kangas, Lars J.; Hayes, James C.; Schrom, Brian T.; Suarez, Reynold; Hubbard, Charles W.; Heimbigner, Tom R.; McIntyre, Justin I.

2009-05-01

Artificial neural networks (ANNs) are used to determine the state-of-health (SOH) of the Automated Radioxenon Analyzer/Sampler (ARSA). ARSA is a gas collection and analysis system used for non-proliferation monitoring in detecting radioxenon released during nuclear tests. SOH diagnostics are important for automated, unmanned sensing systems so that remote detection and identification of problems can be made without onsite staff. Both recurrent and feed-forward ANNs are presented. The recurrent ANN is trained to predict sensor values based on current valve states, which control air flow, so that with only valve states the normal SOH sensor values can be predicted. Deviation between modeled value and actual is an indication of a potential problem. The feed-forward ANN acts as a nonlinear version of principal components analysis (PCA) and is trained to replicate the normal SOH sensor values. Because of ARSA's complexity, this nonlinear PCA is better able to capture the relationships among the sensors than standard linear PCA and is applicable to both sensor validation and recognizing off-normal operating conditions. Both models provide valuable information to detect impending malfunctions before they occur to avoid unscheduled shutdown. Finally, the ability of ANN methods to predict the system state is presented.

Real-Time Monitoring and Fault Diagnosis of a Low Power Hub Motor Using Feedforward Neural Network.

PubMed

Şimşir, Mehmet; Bayır, Raif; Uyaroğlu, Yılmaz

2016-01-01

Low power hub motors are widely used in electromechanical systems such as electrical bicycles and solar vehicles due to their robustness and compact structure. Such systems driven by hub motors (in wheel motors) encounter previously defined and undefined faults under operation. It may inevitably lead to the interruption of the electromechanical system operation; hence, economic losses take place at certain times. Therefore, in order to maintain system operation sustainability, the motor should be precisely monitored and the faults are diagnosed considering various significant motor parameters. In this study, the artificial feedforward backpropagation neural network approach is proposed to real-time monitor and diagnose the faults of the hub motor by measuring seven main system parameters. So as to construct a necessary model, we trained the model, using a data set consisting of 4160 samples where each has 7 parameters, by the MATLAB environment until the best model is obtained. The results are encouraging and meaningful for the specific motor and the developed model may be applicable to other types of hub motors. The prosperous model of the whole system was embedded into Arduino Due microcontroller card and the mobile real-time monitoring and fault diagnosis system prototype for hub motor was designed and manufactured.

Real-Time Monitoring and Fault Diagnosis of a Low Power Hub Motor Using Feedforward Neural Network

PubMed Central

Şimşir, Mehmet; Bayır, Raif; Uyaroğlu, Yılmaz

2016-01-01

Low power hub motors are widely used in electromechanical systems such as electrical bicycles and solar vehicles due to their robustness and compact structure. Such systems driven by hub motors (in wheel motors) encounter previously defined and undefined faults under operation. It may inevitably lead to the interruption of the electromechanical system operation; hence, economic losses take place at certain times. Therefore, in order to maintain system operation sustainability, the motor should be precisely monitored and the faults are diagnosed considering various significant motor parameters. In this study, the artificial feedforward backpropagation neural network approach is proposed to real-time monitor and diagnose the faults of the hub motor by measuring seven main system parameters. So as to construct a necessary model, we trained the model, using a data set consisting of 4160 samples where each has 7 parameters, by the MATLAB environment until the best model is obtained. The results are encouraging and meaningful for the specific motor and the developed model may be applicable to other types of hub motors. The prosperous model of the whole system was embedded into Arduino Due microcontroller card and the mobile real-time monitoring and fault diagnosis system prototype for hub motor was designed and manufactured. PMID:26819590

Ultra-Rapid serial visual presentation reveals dynamics of feedforward and feedback processes in the ventral visual pathway.

PubMed

Mohsenzadeh, Yalda; Qin, Sheng; Cichy, Radoslaw M; Pantazis, Dimitrios

2018-06-21

Human visual recognition activates a dense network of overlapping feedforward and recurrent neuronal processes, making it hard to disentangle processing in the feedforward from the feedback direction. Here, we used ultra-rapid serial visual presentation to suppress sustained activity that blurs the boundaries of processing steps, enabling us to resolve two distinct stages of processing with MEG multivariate pattern classification. The first processing stage was the rapid activation cascade of the bottom-up sweep, which terminated early as visual stimuli were presented at progressively faster rates. The second stage was the emergence of categorical information with peak latency that shifted later in time with progressively faster stimulus presentations, indexing time-consuming recurrent processing. Using MEG-fMRI fusion with representational similarity, we localized recurrent signals in early visual cortex. Together, our findings segregated an initial bottom-up sweep from subsequent feedback processing, and revealed the neural signature of increased recurrent processing demands for challenging viewing conditions. © 2018, Mohsenzadeh et al.

Min-max hyperellipsoidal clustering for anomaly detection in network security.

PubMed

Sarasamma, Suseela T; Zhu, Qiuming A

2006-08-01

A novel hyperellipsoidal clustering technique is presented for an intrusion-detection system in network security. Hyperellipsoidal clusters toward maximum intracluster similarity and minimum intercluster similarity are generated from training data sets. The novelty of the technique lies in the fact that the parameters needed to construct higher order data models in general multivariate Gaussian functions are incrementally derived from the data sets using accretive processes. The technique is implemented in a feedforward neural network that uses a Gaussian radial basis function as the model generator. An evaluation based on the inclusiveness and exclusiveness of samples with respect to specific criteria is applied to accretively learn the output clusters of the neural network. One significant advantage of this is its ability to detect individual anomaly types that are hard to detect with other anomaly-detection schemes. Applying this technique, several feature subsets of the tcptrace network-connection records that give above 95% detection at false-positive rates below 5% were identified.

An Approach to Stable Gradient-Descent Adaptation of Higher Order Neural Units.

PubMed

Bukovsky, Ivo; Homma, Noriyasu

2017-09-01

Stability evaluation of a weight-update system of higher order neural units (HONUs) with polynomial aggregation of neural inputs (also known as classes of polynomial neural networks) for adaptation of both feedforward and recurrent HONUs by a gradient descent method is introduced. An essential core of the approach is based on the spectral radius of a weight-update system, and it allows stability monitoring and its maintenance at every adaptation step individually. Assuring the stability of the weight-update system (at every single adaptation step) naturally results in the adaptation stability of the whole neural architecture that adapts to the target data. As an aside, the used approach highlights the fact that the weight optimization of HONU is a linear problem, so the proposed approach can be generally extended to any neural architecture that is linear in its adaptable parameters.

A real time neural net estimator of fatigue life

NASA Technical Reports Server (NTRS)

Troudet, T.; Merrill, W.

1990-01-01

A neural network architecture is proposed to estimate, in real-time, the fatigue life of mechanical components, as part of the intelligent Control System for Reusable Rocket Engines. Arbitrary component loading values were used as input to train a two hidden-layer feedforward neural net to estimate component fatigue damage. The ability of the net to learn, based on a local strain approach, the mapping between load sequence and fatigue damage has been demonstrated for a uniaxial specimen. Because of its demonstrated performance, the neural computation may be extended to complex cases where the loads are biaxial or triaxial, and the geometry of the component is complex (e.g., turbopumps blades). The generality of the approach is such that load/damage mappings can be directly extracted from experimental data without requiring any knowledge of the stress/strain profile of the component. In addition, the parallel network architecture allows real-time life calculations even for high-frequency vibrations. Owing to its distributed nature, the neural implementation will be robust and reliable, enabling its use in hostile environments such as rocket engines.

Mirrored STDP Implements Autoencoder Learning in a Network of Spiking Neurons.

PubMed

Burbank, Kendra S

2015-12-01

The autoencoder algorithm is a simple but powerful unsupervised method for training neural networks. Autoencoder networks can learn sparse distributed codes similar to those seen in cortical sensory areas such as visual area V1, but they can also be stacked to learn increasingly abstract representations. Several computational neuroscience models of sensory areas, including Olshausen & Field's Sparse Coding algorithm, can be seen as autoencoder variants, and autoencoders have seen extensive use in the machine learning community. Despite their power and versatility, autoencoders have been difficult to implement in a biologically realistic fashion. The challenges include their need to calculate differences between two neuronal activities and their requirement for learning rules which lead to identical changes at feedforward and feedback connections. Here, we study a biologically realistic network of integrate-and-fire neurons with anatomical connectivity and synaptic plasticity that closely matches that observed in cortical sensory areas. Our choice of synaptic plasticity rules is inspired by recent experimental and theoretical results suggesting that learning at feedback connections may have a different form from learning at feedforward connections, and our results depend critically on this novel choice of plasticity rules. Specifically, we propose that plasticity rules at feedforward versus feedback connections are temporally opposed versions of spike-timing dependent plasticity (STDP), leading to a symmetric combined rule we call Mirrored STDP (mSTDP). We show that with mSTDP, our network follows a learning rule that approximately minimizes an autoencoder loss function. When trained with whitened natural image patches, the learned synaptic weights resemble the receptive fields seen in V1. Our results use realistic synaptic plasticity rules to show that the powerful autoencoder learning algorithm could be within the reach of real biological networks.

Mirrored STDP Implements Autoencoder Learning in a Network of Spiking Neurons

PubMed Central

Burbank, Kendra S.

2015-01-01

The autoencoder algorithm is a simple but powerful unsupervised method for training neural networks. Autoencoder networks can learn sparse distributed codes similar to those seen in cortical sensory areas such as visual area V1, but they can also be stacked to learn increasingly abstract representations. Several computational neuroscience models of sensory areas, including Olshausen & Field’s Sparse Coding algorithm, can be seen as autoencoder variants, and autoencoders have seen extensive use in the machine learning community. Despite their power and versatility, autoencoders have been difficult to implement in a biologically realistic fashion. The challenges include their need to calculate differences between two neuronal activities and their requirement for learning rules which lead to identical changes at feedforward and feedback connections. Here, we study a biologically realistic network of integrate-and-fire neurons with anatomical connectivity and synaptic plasticity that closely matches that observed in cortical sensory areas. Our choice of synaptic plasticity rules is inspired by recent experimental and theoretical results suggesting that learning at feedback connections may have a different form from learning at feedforward connections, and our results depend critically on this novel choice of plasticity rules. Specifically, we propose that plasticity rules at feedforward versus feedback connections are temporally opposed versions of spike-timing dependent plasticity (STDP), leading to a symmetric combined rule we call Mirrored STDP (mSTDP). We show that with mSTDP, our network follows a learning rule that approximately minimizes an autoencoder loss function. When trained with whitened natural image patches, the learned synaptic weights resemble the receptive fields seen in V1. Our results use realistic synaptic plasticity rules to show that the powerful autoencoder learning algorithm could be within the reach of real biological networks. PMID:26633645

From Molecular Circuit Dysfunction to Disease: Case Studies in Epilepsy, Traumatic Brain Injury, and Alzheimer’s Disease

PubMed Central

Dulla, Chris G.; Coulter, Douglas A.; Ziburkus, Jokubas

2015-01-01

Complex circuitry with feed-forward and feed-back systems regulate neuronal activity throughout the brain. Cell biological, electrical, and neurotransmitter systems enable neural networks to process and drive the entire spectrum of cognitive, behavioral, and motor functions. Simultaneous orchestration of distinct cells and interconnected neural circuits relies on hundreds, if not thousands, of unique molecular interactions. Even single molecule dysfunctions can be disrupting to neural circuit activity, leading to neurological pathology. Here, we sample our current understanding of how molecular aberrations lead to disruptions in networks using three neurological pathologies as exemplars: epilepsy, traumatic brain injury (TBI), and Alzheimer’s disease (AD). Epilepsy provides a window into how total destabilization of network balance can occur. TBI is an abrupt physical disruption that manifests in both acute and chronic neurological deficits. Last, in AD progressive cell loss leads to devastating cognitive consequences. Interestingly, all three of these neurological diseases are interrelated. The goal of this review, therefore, is to identify molecular changes that may lead to network dysfunction, elaborate on how altered network activity and circuit structure can contribute to neurological disease, and suggest common threads that may lie at the heart of molecular circuit dysfunction. PMID:25948650

Optimizing a neural network for detection of moving vehicles in video

NASA Astrophysics Data System (ADS)

Fischer, Noëlle M.; Kruithof, Maarten C.; Bouma, Henri

2017-10-01

In the field of security and defense, it is extremely important to reliably detect moving objects, such as cars, ships, drones and missiles. Detection and analysis of moving objects in cameras near borders could be helpful to reduce illicit trading, drug trafficking, irregular border crossing, trafficking in human beings and smuggling. Many recent benchmarks have shown that convolutional neural networks are performing well in the detection of objects in images. Most deep-learning research effort focuses on classification or detection on single images. However, the detection of dynamic changes (e.g., moving objects, actions and events) in streaming video is extremely relevant for surveillance and forensic applications. In this paper, we combine an end-to-end feedforward neural network for static detection with a recurrent Long Short-Term Memory (LSTM) network for multi-frame analysis. We present a practical guide with special attention to the selection of the optimizer and batch size. The end-to-end network is able to localize and recognize the vehicles in video from traffic cameras. We show an efficient way to collect relevant in-domain data for training with minimal manual labor. Our results show that the combination with LSTM improves performance for the detection of moving vehicles.

A Software Package for Neural Network Applications Development

NASA Technical Reports Server (NTRS)

Baran, Robert H.

1993-01-01

Original Backprop (Version 1.2) is an MS-DOS package of four stand-alone C-language programs that enable users to develop neural network solutions to a variety of practical problems. Original Backprop generates three-layer, feed-forward (series-coupled) networks which map fixed-length input vectors into fixed length output vectors through an intermediate (hidden) layer of binary threshold units. Version 1.2 can handle up to 200 input vectors at a time, each having up to 128 real-valued components. The first subprogram, TSET, appends a number (up to 16) of classification bits to each input, thus creating a training set of input output pairs. The second subprogram, BACKPROP, creates a trilayer network to do the prescribed mapping and modifies the weights of its connections incrementally until the training set is leaned. The learning algorithm is the 'back-propagating error correction procedures first described by F. Rosenblatt in 1961. The third subprogram, VIEWNET, lets the trained network be examined, tested, and 'pruned' (by the deletion of unnecessary hidden units). The fourth subprogram, DONET, makes a TSR routine by which the finished product of the neural net design-and-training exercise can be consulted under other MS-DOS applications.

From Molecular Circuit Dysfunction to Disease: Case Studies in Epilepsy, Traumatic Brain Injury, and Alzheimer's Disease.

PubMed

Dulla, Chris G; Coulter, Douglas A; Ziburkus, Jokubas

2016-06-01

Complex circuitry with feed-forward and feed-back systems regulate neuronal activity throughout the brain. Cell biological, electrical, and neurotransmitter systems enable neural networks to process and drive the entire spectrum of cognitive, behavioral, and motor functions. Simultaneous orchestration of distinct cells and interconnected neural circuits relies on hundreds, if not thousands, of unique molecular interactions. Even single molecule dysfunctions can be disrupting to neural circuit activity, leading to neurological pathology. Here, we sample our current understanding of how molecular aberrations lead to disruptions in networks using three neurological pathologies as exemplars: epilepsy, traumatic brain injury (TBI), and Alzheimer's disease (AD). Epilepsy provides a window into how total destabilization of network balance can occur. TBI is an abrupt physical disruption that manifests in both acute and chronic neurological deficits. Last, in AD progressive cell loss leads to devastating cognitive consequences. Interestingly, all three of these neurological diseases are interrelated. The goal of this review, therefore, is to identify molecular changes that may lead to network dysfunction, elaborate on how altered network activity and circuit structure can contribute to neurological disease, and suggest common threads that may lie at the heart of molecular circuit dysfunction. © The Author(s) 2015.

Hybrid feedback feedforward: An efficient design of adaptive neural network control.

PubMed

Pan, Yongping; Liu, Yiqi; Xu, Bin; Yu, Haoyong

2016-04-01

This paper presents an efficient hybrid feedback feedforward (HFF) adaptive approximation-based control (AAC) strategy for a class of uncertain Euler-Lagrange systems. The control structure includes a proportional-derivative (PD) control term in the feedback loop and a radial-basis-function (RBF) neural network (NN) in the feedforward loop, which mimics the human motor learning control mechanism. At the presence of discontinuous friction, a sigmoid-jump-function NN is incorporated to improve control performance. The major difference of the proposed HFF-AAC design from the traditional feedback AAC (FB-AAC) design is that only desired outputs, rather than both tracking errors and desired outputs, are applied as RBF-NN inputs. Yet, such a slight modification leads to several attractive properties of HFF-AAC, including the convenient choice of an approximation domain, the decrease of the number of RBF-NN inputs, and semiglobal practical asymptotic stability dominated by control gains. Compared with previous HFF-AAC approaches, the proposed approach possesses the following two distinctive features: (i) all above attractive properties are achieved by a much simpler control scheme; (ii) the bounds of plant uncertainties are not required to be known. Consequently, the proposed approach guarantees a minimum configuration of the control structure and a minimum requirement of plant knowledge for the AAC design, which leads to a sharp decrease of implementation cost in terms of hardware selection, algorithm realization and system debugging. Simulation results have demonstrated that the proposed HFF-AAC can perform as good as or even better than the traditional FB-AAC under much simpler control synthesis and much lower computational cost. Copyright © 2015 Elsevier Ltd. All rights reserved.

Quantitative workflow based on NN for weighting criteria in landfill suitability mapping

NASA Astrophysics Data System (ADS)

Abujayyab, Sohaib K. M.; Ahamad, Mohd Sanusi S.; Yahya, Ahmad Shukri; Ahmad, Siti Zubaidah; Alkhasawneh, Mutasem Sh.; Aziz, Hamidi Abdul

2017-10-01

Our study aims to introduce a new quantitative workflow that integrates neural networks (NNs) and multi criteria decision analysis (MCDA). Existing MCDA workflows reveal a number of drawbacks, because of the reliance on human knowledge in the weighting stage. Thus, new workflow presented to form suitability maps at the regional scale for solid waste planning based on NNs. A feed-forward neural network employed in the workflow. A total of 34 criteria were pre-processed to establish the input dataset for NN modelling. The final learned network used to acquire the weights of the criteria. Accuracies of 95.2% and 93.2% achieved for the training dataset and testing dataset, respectively. The workflow was found to be capable of reducing human interference to generate highly reliable maps. The proposed workflow reveals the applicability of NN in generating landfill suitability maps and the feasibility of integrating them with existing MCDA workflows.

Adaptive Neural Network Control of a Flapping Wing Micro Aerial Vehicle With Disturbance Observer.

PubMed

He, Wei; Yan, Zichen; Sun, Changyin; Chen, Yunan

2017-10-01

The research of this paper works out the attitude and position control of the flapping wing micro aerial vehicle (FWMAV). Neural network control with full state and output feedback are designed to deal with uncertainties in this complex nonlinear FWMAV dynamic system and enhance the system robustness. Meanwhile, we design disturbance observers which are exerted into the FWMAV system via feedforward loops to counteract the bad influence of disturbances. Then, a Lyapunov function is proposed to prove the closed-loop system stability and the semi-global uniform ultimate boundedness of all state variables. Finally, a series of simulation results indicate that proposed controllers can track desired trajectories well via selecting appropriate control gains. And the designed controllers possess potential applications in FWMAVs.

Initial results on fault diagnosis of DSN antenna control assemblies using pattern recognition techniques

NASA Technical Reports Server (NTRS)

Smyth, P.; Mellstrom, J.

1990-01-01

Initial results obtained from an investigation using pattern recognition techniques for identifying fault modes in the Deep Space Network (DSN) 70 m antenna control loops are described. The overall background to the problem is described, the motivation and potential benefits of this approach are outlined. In particular, an experiment is described in which fault modes were introduced into a state-space simulation of the antenna control loops. By training a multilayer feed-forward neural network on the simulated sensor output, classification rates of over 95 percent were achieved with a false alarm rate of zero on unseen tests data. It concludes that although the neural classifier has certain practical limitations at present, it also has considerable potential for problems of this nature.

Strawberry Maturity Neural Network Detectng System Based on Genetic Algorithm

NASA Astrophysics Data System (ADS)

Xu, Liming

The quick and non-detective detection of agriculture product is one of the measures to increase the precision and productivity of harvesting and grading. Having analyzed H frequency of different maturities in different light intensities, the results show that H frequency for the same maturity has little influence in different light intensities; Under the same light intensity, three strawberry maturities are changing in order. After having confirmed the H frequency section to distinguish the different strawberry maturity, the triplelayer feed-forward neural network system to detect strawberry maturity was designed by using genetic algorithm. The test results show that the detecting precision ratio is 91.7%, it takes 160ms to distinguish one strawberry. Therefore, the online non-detective detecting the strawberry maturity could be realized.

Investigation of bus transit schedule behavior modeling using advanced techniques

DOE Office of Scientific and Technical Information (OSTI.GOV)

Kalaputapu, R.; Demetsky, M.J.

This research focused on investigating the application of artificial neural networks (ANN) and the Box-Jenkins technique for developing and testing schedule behavior models using data obtained for a test route from Tidewater Regional Transit`s AVL system. The three ANN architectures investigated were: Feedforward Network, Elman Network and Jordan Network. In addition, five different model structures were investigated. The time-series methodology was adopted for developing the schedule behavior models. Finally, the role of a schedule behavior model within the framework of an intelligent transit management system is defined and the potential utility of the schedule behavior model is discussed using anmore » example application.« less

Prediction of β-turns in proteins from multiple alignment using neural network

PubMed Central

Kaur, Harpreet; Raghava, Gajendra Pal Singh

2003-01-01

A neural network-based method has been developed for the prediction of β-turns in proteins by using multiple sequence alignment. Two feed-forward back-propagation networks with a single hidden layer are used where the first-sequence structure network is trained with the multiple sequence alignment in the form of PSI-BLAST–generated position-specific scoring matrices. The initial predictions from the first network and PSIPRED-predicted secondary structure are used as input to the second structure-structure network to refine the predictions obtained from the first net. A significant improvement in prediction accuracy has been achieved by using evolutionary information contained in the multiple sequence alignment. The final network yields an overall prediction accuracy of 75.5% when tested by sevenfold cross-validation on a set of 426 nonhomologous protein chains. The corresponding Qpred, Qobs, and Matthews correlation coefficient values are 49.8%, 72.3%, and 0.43, respectively, and are the best among all the previously published β-turn prediction methods. The Web server BetaTPred2 (http://www.imtech.res.in/raghava/betatpred2/) has been developed based on this approach. PMID:12592033

Neural control of magnetic suspension systems

NASA Technical Reports Server (NTRS)

Gray, W. Steven

1993-01-01

The purpose of this research program is to design, build and test (in cooperation with NASA personnel from the NASA Langley Research Center) neural controllers for two different small air-gap magnetic suspension systems. The general objective of the program is to study neural network architectures for the purpose of control in an experimental setting and to demonstrate the feasibility of the concept. The specific objectives of the research program are: (1) to demonstrate through simulation and experimentation the feasibility of using neural controllers to stabilize a nonlinear magnetic suspension system; (2) to investigate through simulation and experimentation the performance of neural controllers designs under various types of parametric and nonparametric uncertainty; (3) to investigate through simulation and experimentation various types of neural architectures for real-time control with respect to performance and complexity; and (4) to benchmark in an experimental setting the performance of neural controllers against other types of existing linear and nonlinear compensator designs. To date, the first one-dimensional, small air-gap magnetic suspension system has been built, tested and delivered to the NASA Langley Research Center. The device is currently being stabilized with a digital linear phase-lead controller. The neural controller hardware is under construction. Two different neural network paradigms are under consideration, one based on hidden layer feedforward networks trained via back propagation and one based on using Gaussian radial basis functions trained by analytical methods related to stability conditions. Some advanced nonlinear control algorithms using feedback linearization and sliding mode control are in simulation studies.

Theory of cortical function

PubMed Central

Heeger, David J.

2017-01-01

Most models of sensory processing in the brain have a feedforward architecture in which each stage comprises simple linear filtering operations and nonlinearities. Models of this form have been used to explain a wide range of neurophysiological and psychophysical data, and many recent successes in artificial intelligence (with deep convolutional neural nets) are based on this architecture. However, neocortex is not a feedforward architecture. This paper proposes a first step toward an alternative computational framework in which neural activity in each brain area depends on a combination of feedforward drive (bottom-up from the previous processing stage), feedback drive (top-down context from the next stage), and prior drive (expectation). The relative contributions of feedforward drive, feedback drive, and prior drive are controlled by a handful of state parameters, which I hypothesize correspond to neuromodulators and oscillatory activity. In some states, neural responses are dominated by the feedforward drive and the theory is identical to a conventional feedforward model, thereby preserving all of the desirable features of those models. In other states, the theory is a generative model that constructs a sensory representation from an abstract representation, like memory recall. In still other states, the theory combines prior expectation with sensory input, explores different possible perceptual interpretations of ambiguous sensory inputs, and predicts forward in time. The theory, therefore, offers an empirically testable framework for understanding how the cortex accomplishes inference, exploration, and prediction. PMID:28167793

Detecting Industrial Chemicals in Water With Microbial Fuel Cells and Artificial Neural Networks

DTIC Science & Technology

2014-03-27

aquifers are all examples of prominent concerns regarding water quality degradation today. Water quality monitoring to detect contamination issues is...chitin. At the cathode, also known as a biocathode, electrons are used by bacteria to reduce the contaminants (Logan, 2008). Gregory et al. (2004...forecasting of water quality index for the Kinta River in Malaysia . They used multiple water quality parameters as inputs in a feed-forward ANN with

Artificial Neural Networks-Based Software for Measuring Heat Collection Rate and Heat Loss Coefficient of Water-in-Glass Evacuated Tube Solar Water Heaters

PubMed Central

Liu, Zhijian; Liu, Kejun; Li, Hao; Zhang, Xinyu; Jin, Guangya; Cheng, Kewei

2015-01-01

Measurements of heat collection rate and heat loss coefficient are crucial for the evaluation of in service water-in-glass evacuated tube solar water heaters. However, conventional measurement requires expensive detection devices and undergoes a series of complicated procedures. To simplify the measurement and reduce the cost, software based on artificial neural networks for measuring heat collection rate and heat loss coefficient of water-in-glass evacuated tube solar water heaters was developed. Using multilayer feed-forward neural networks with back-propagation algorithm, we developed and tested our program on the basis of 915measuredsamples of water-in-glass evacuated tube solar water heaters. This artificial neural networks-based software program automatically obtained accurate heat collection rateand heat loss coefficient using simply "portable test instruments" acquired parameters, including tube length, number of tubes, tube center distance, heat water mass in tank, collector area, angle between tubes and ground and final temperature. Our results show that this software (on both personal computer and Android platforms) is efficient and convenient to predict the heat collection rate and heat loss coefficient due to it slow root mean square errors in prediction. The software now can be downloaded from http://t.cn/RLPKF08. PMID:26624613

Artificial Neural Networks-Based Software for Measuring Heat Collection Rate and Heat Loss Coefficient of Water-in-Glass Evacuated Tube Solar Water Heaters.

PubMed

Liu, Zhijian; Liu, Kejun; Li, Hao; Zhang, Xinyu; Jin, Guangya; Cheng, Kewei

2015-01-01

Measurements of heat collection rate and heat loss coefficient are crucial for the evaluation of in service water-in-glass evacuated tube solar water heaters. However, conventional measurement requires expensive detection devices and undergoes a series of complicated procedures. To simplify the measurement and reduce the cost, software based on artificial neural networks for measuring heat collection rate and heat loss coefficient of water-in-glass evacuated tube solar water heaters was developed. Using multilayer feed-forward neural networks with back-propagation algorithm, we developed and tested our program on the basis of 915 measured samples of water-in-glass evacuated tube solar water heaters. This artificial neural networks-based software program automatically obtained accurate heat collection rate and heat loss coefficient using simply "portable test instruments" acquired parameters, including tube length, number of tubes, tube center distance, heat water mass in tank, collector area, angle between tubes and ground and final temperature. Our results show that this software (on both personal computer and Android platforms) is efficient and convenient to predict the heat collection rate and heat loss coefficient due to it slow root mean square errors in prediction. The software now can be downloaded from http://t.cn/RLPKF08.

Near real-time analysis of extrinsic Fabry-Perot interferometric sensors under damped vibration using artificial neural networks

NASA Astrophysics Data System (ADS)

Dua, Rohit; Watkins, Steve E.

2009-03-01

Strain analysis due to vibration can provide insight into structural health. An Extrinsic Fabry-Perot Interferometric (EFPI) sensor under vibrational strain generates a non-linear modulated output. Advanced signal processing techniques, to extract important information such as absolute strain, are required to demodulate this non-linear output. Past research has employed Artificial Neural Networks (ANN) and Fast Fourier Transforms (FFT) to demodulate the EFPI sensor for limited conditions. These demodulation systems could only handle variations in absolute value of strain and frequency of actuation during a vibration event. This project uses an ANN approach to extend the demodulation system to include the variation in the damping coefficient of the actuating vibration, in a near real-time vibration scenario. A computer simulation provides training and testing data for the theoretical output of the EFPI sensor to demonstrate the approaches. FFT needed to be performed on a window of the EFPI output data. A small window of observation is obtained, while maintaining low absolute-strain prediction errors, heuristically. Results are obtained and compared from employing different ANN architectures including multi-layered feedforward ANN trained using Backpropagation Neural Network (BPNN), and Generalized Regression Neural Networks (GRNN). A two-layered algorithm fusion system is developed and tested that yields better results.

The Role of Architectural and Learning Constraints in Neural Network Models: A Case Study on Visual Space Coding.

PubMed

Testolin, Alberto; De Filippo De Grazia, Michele; Zorzi, Marco

2017-01-01

The recent "deep learning revolution" in artificial neural networks had strong impact and widespread deployment for engineering applications, but the use of deep learning for neurocomputational modeling has been so far limited. In this article we argue that unsupervised deep learning represents an important step forward for improving neurocomputational models of perception and cognition, because it emphasizes the role of generative learning as opposed to discriminative (supervised) learning. As a case study, we present a series of simulations investigating the emergence of neural coding of visual space for sensorimotor transformations. We compare different network architectures commonly used as building blocks for unsupervised deep learning by systematically testing the type of receptive fields and gain modulation developed by the hidden neurons. In particular, we compare Restricted Boltzmann Machines (RBMs), which are stochastic, generative networks with bidirectional connections trained using contrastive divergence, with autoencoders, which are deterministic networks trained using error backpropagation. For both learning architectures we also explore the role of sparse coding, which has been identified as a fundamental principle of neural computation. The unsupervised models are then compared with supervised, feed-forward networks that learn an explicit mapping between different spatial reference frames. Our simulations show that both architectural and learning constraints strongly influenced the emergent coding of visual space in terms of distribution of tuning functions at the level of single neurons. Unsupervised models, and particularly RBMs, were found to more closely adhere to neurophysiological data from single-cell recordings in the primate parietal cortex. These results provide new insights into how basic properties of artificial neural networks might be relevant for modeling neural information processing in biological systems.

The Role of Architectural and Learning Constraints in Neural Network Models: A Case Study on Visual Space Coding

PubMed Central

Testolin, Alberto; De Filippo De Grazia, Michele; Zorzi, Marco

2017-01-01

The recent “deep learning revolution” in artificial neural networks had strong impact and widespread deployment for engineering applications, but the use of deep learning for neurocomputational modeling has been so far limited. In this article we argue that unsupervised deep learning represents an important step forward for improving neurocomputational models of perception and cognition, because it emphasizes the role of generative learning as opposed to discriminative (supervised) learning. As a case study, we present a series of simulations investigating the emergence of neural coding of visual space for sensorimotor transformations. We compare different network architectures commonly used as building blocks for unsupervised deep learning by systematically testing the type of receptive fields and gain modulation developed by the hidden neurons. In particular, we compare Restricted Boltzmann Machines (RBMs), which are stochastic, generative networks with bidirectional connections trained using contrastive divergence, with autoencoders, which are deterministic networks trained using error backpropagation. For both learning architectures we also explore the role of sparse coding, which has been identified as a fundamental principle of neural computation. The unsupervised models are then compared with supervised, feed-forward networks that learn an explicit mapping between different spatial reference frames. Our simulations show that both architectural and learning constraints strongly influenced the emergent coding of visual space in terms of distribution of tuning functions at the level of single neurons. Unsupervised models, and particularly RBMs, were found to more closely adhere to neurophysiological data from single-cell recordings in the primate parietal cortex. These results provide new insights into how basic properties of artificial neural networks might be relevant for modeling neural information processing in biological systems. PMID:28377709

Application of a feedforward neural network in the search for kuroko deposits in the Hokuroku District, Japan

USGS Publications Warehouse

Singer, Donald A.; Kouda, Ryoichi

1996-01-01

A feedforward neural network with one hidden layer and five neurons was trained to recognize the distance to kuroko mineral deposits. Average amounts per hole of pyrite, sericite, and gypsum plus anhydrite as measured by X-rays in 69 drillholes were used to train the net. Drillholes near and between the Fukazawa, Furutobe, and Shakanai mines were used. The training data were selected carefully to represent well-explored areas where some confidence of the distance to ore was assured. A logarithmic transform was applied to remove the skewness of distance and each variable was scaled and centered by subtracting the median and dividing by the interquartile range. The learning algorithm of annealing plus conjugate gradients was used to minimize the mean squared error of the scaled distance to ore. The trained network then was applied to all of the 152 drillholes that had measured gypsum, sericite, and pyrite. A contour plot of the neural net predicted distance to ore shows fairly wide areas of 1 km or less to ore; each of the known deposit groups is within the 1 km contour. The high and low distances on the margins of the contoured distance plot are in part the result of boundary effects of the contouring algorithm. For example, the short distances to ore predicted west of the Shakanai (Hanaoka) deposits are in basement. However, the short distances to ore predicted northeast of Furotobe, just off the figure, coincide with the location of the Nurukawa kuroko deposit and the Omaki deposit, south of the Shakanai-Hanaoka deposits, seems to be on an extension of short distance to ore contour, but is beyond the 3 km limit from drillholes. Also of interest are some areas only a few kilometers from the Fukazawa and Shakanai groups of deposits that are estimated to be many kilometers from ore, apparently reflecting the network's recognition of the extreme local variability of the geology near some deposits.

Random noise effects in pulse-mode digital multilayer neural networks.

PubMed

Kim, Y C; Shanblatt, M A

1995-01-01

A pulse-mode digital multilayer neural network (DMNN) based on stochastic computing techniques is implemented with simple logic gates as basic computing elements. The pulse-mode signal representation and the use of simple logic gates for neural operations lead to a massively parallel yet compact and flexible network architecture, well suited for VLSI implementation. Algebraic neural operations are replaced by stochastic processes using pseudorandom pulse sequences. The distributions of the results from the stochastic processes are approximated using the hypergeometric distribution. Synaptic weights and neuron states are represented as probabilities and estimated as average pulse occurrence rates in corresponding pulse sequences. A statistical model of the noise (error) is developed to estimate the relative accuracy associated with stochastic computing in terms of mean and variance. Computational differences are then explained by comparison to deterministic neural computations. DMNN feedforward architectures are modeled in VHDL using character recognition problems as testbeds. Computational accuracy is analyzed, and the results of the statistical model are compared with the actual simulation results. Experiments show that the calculations performed in the DMNN are more accurate than those anticipated when Bernoulli sequences are assumed, as is common in the literature. Furthermore, the statistical model successfully predicts the accuracy of the operations performed in the DMNN.

Functional electrical stimulation controlled by artificial neural networks: pilot experiments with simple movements are promising for rehabilitation applications.

PubMed

Ferrante, Simona; Pedrocchi, Alessandra; Iannò, Marco; De Momi, Elena; Ferrarin, Maurizio; Ferrigno, Giancarlo

2004-01-01

This study falls within the ambit of research on functional electrical stimulation for the design of rehabilitation training for spinal cord injured patients. In this context, a crucial issue is the control of the stimulation parameters in order to optimize the patterns of muscle activation and to increase the duration of the exercises. An adaptive control system (NEURADAPT) based on artificial neural networks (ANNs) was developed to control the knee joint in accordance with desired trajectories by stimulating quadriceps muscles. This strategy includes an inverse neural model of the stimulated limb in the feedforward line and a neural network trained on-line in the feedback loop. NEURADAPT was compared with a linear closed-loop proportional integrative derivative (PID) controller and with a model-based neural controller (NEUROPID). Experiments on two subjects (one healthy and one paraplegic) show the good performance of NEURADAPT, which is able to reduce the time lag introduced by the PID controller. In addition, control systems based on ANN techniques do not require complicated calibration procedures at the beginning of each experimental session. After the initial learning phase, the ANN, thanks to its generalization capacity, is able to cope with a certain range of variability of skeletal muscle properties.

Application of artificial neural network to fMRI regression analysis.

PubMed

Misaki, Masaya; Miyauchi, Satoru

2006-01-15

We used an artificial neural network (ANN) to detect correlations between event sequences and fMRI (functional magnetic resonance imaging) signals. The layered feed-forward neural network, given a series of events as inputs and the fMRI signal as a supervised signal, performed a non-linear regression analysis. This type of ANN is capable of approximating any continuous function, and thus this analysis method can detect any fMRI signals that correlated with corresponding events. Because of the flexible nature of ANNs, fitting to autocorrelation noise is a problem in fMRI analyses. We avoided this problem by using cross-validation and an early stopping procedure. The results showed that the ANN could detect various responses with different time courses. The simulation analysis also indicated an additional advantage of ANN over non-parametric methods in detecting parametrically modulated responses, i.e., it can detect various types of parametric modulations without a priori assumptions. The ANN regression analysis is therefore beneficial for exploratory fMRI analyses in detecting continuous changes in responses modulated by changes in input values.

Modeling and simulation of xylitol production in bioreactor by Debaryomyces nepalensis NCYC 3413 using unstructured and artificial neural network models.

PubMed

Pappu, J Sharon Mano; Gummadi, Sathyanarayana N

2016-11-01

This study examines the use of unstructured kinetic model and artificial neural networks as predictive tools for xylitol production by Debaryomyces nepalensis NCYC 3413 in bioreactor. An unstructured kinetic model was proposed in order to assess the influence of pH (4, 5 and 6), temperature (25°C, 30°C and 35°C) and volumetric oxygen transfer coefficient kLa (0.14h(-1), 0.28h(-1) and 0.56h(-1)) on growth and xylitol production. A feed-forward back-propagation artificial neural network (ANN) has been developed to investigate the effect of process condition on xylitol production. ANN configuration of 6-10-3 layers was selected and trained with 339 experimental data points from bioreactor studies. Results showed that simulation and prediction accuracy of ANN was apparently higher when compared to unstructured mechanistic model under varying operational conditions. ANN was found to be an efficient data-driven tool to predict the optimal harvest time in xylitol production. Copyright © 2016 Elsevier Ltd. All rights reserved.

Predicting the spatial distribution of soil profile in Adapazari/Turkey by artificial neural networks using CPT data

NASA Astrophysics Data System (ADS)

Arel, Ersin

2012-06-01

The infamous soils of Adapazari, Turkey, that failed extensively during the 46-s long magnitude 7.4 earthquake in 1999 have since been the subject of a research program. Boreholes, piezocone soundings and voluminous laboratory testing have enabled researchers to apply sophisticated methods to determine the soil profiles in the city using the existing database. This paper describes the use of the artificial neural network (ANN) model to predict the complex soil profiles of Adapazari, based on cone penetration test (CPT) results. More than 3236 field CPT readings have been collected from 117 soundings spread over an area of 26 km2. An attempt has been made to develop the ANN model using multilayer perceptrons trained with a feed-forward back-propagation algorithm. The results show that the ANN model is fairly accurate in predicting complex soil profiles. Soil identification using CPT test results has principally been based on the Robertson charts. Applying neural network systems using the chart offers a powerful and rapid route to reliable prediction of the soil profiles.

Mullite ceramic membranes for industrial oily wastewater treatment: experimental and neural network modeling.

PubMed

Shokrkar, H; Salahi, A; Kasiri, N; Mohammadi, T

2011-01-01

In this paper, results of an experimental and modeling of separation of oil from industrial oily wastewaters (desalter unit effluent of Seraje, Ghom gas wells, Iran) with mullite ceramic membranes are presented. Mullite microfiltration symmetric membranes were synthesized from kaolin clay and alpha-alumina powder. The results show that the mullite ceramic membrane has a high total organic carbon and chemical oxygen demand rejection (94 and 89%, respectively), a low fouling resistance (30%) and a high final permeation flux (75 L/m2 h). Also, an artificial neural network, a predictive tool for tracking the inputs and outputs of a non-linear problem, is used to model the permeation flux decline during microfiltration of oily wastewater. The aim was to predict the permeation flux as a function of feed temperature, trans-membrane pressure, cross-flow velocity, oil concentration and filtration time, using a feed-forward neural network. Finally the structure of hidden layers and nodes in each layer with minimum error were reported leading to a 4-15 structure which demonstrated good agreement with the experimental measurements with an average error of less than 2%.

Contact-force distribution optimization and control for quadruped robots using both gradient and adaptive neural networks.

PubMed

Li, Zhijun; Ge, Shuzhi Sam; Liu, Sibang

2014-08-01

This paper investigates optimal feet forces' distribution and control of quadruped robots under external disturbance forces. First, we formulate a constrained dynamics of quadruped robots and derive a reduced-order dynamical model of motion/force. Consider an external wrench on quadruped robots; the distribution of required forces and moments on the supporting legs of a quadruped robot is handled as a tip-point force distribution and used to equilibrate the external wrench. Then, a gradient neural network is adopted to deal with the optimized objective function formulated as to minimize this quadratic objective function subjected to linear equality and inequality constraints. For the obtained optimized tip-point force and the motion of legs, we propose the hybrid motion/force control based on an adaptive neural network to compensate for the perturbations in the environment and approximate feedforward force and impedance of the leg joints. The proposed control can confront the uncertainties including approximation error and external perturbation. The verification of the proposed control is conducted using a simulation.

A neural network model for predicting weighted mean temperature

NASA Astrophysics Data System (ADS)

Ding, Maohua

2018-02-01

Water vapor is an important element of the Earth's atmosphere, and most of it concentrates at the bottom of the troposphere. Knowledge of the water vapor measured by Global Navigation Satellite Systems (GNSS) is an important direction of GNSS research. In particular, when the zenith wet delay is converted to precipitable water vapor, the weighted mean temperature T_m is a variable parameter to be determined in this conversion. The purpose of the study is getting a more accurate T_m model for global users by a combination of two different characteristics of T_m (i.e., the T_m seasonal variations and the relationships between T_m and surface meteorological elements). The modeling process was carried out by using the neural network technology. A multilayer feedforward neural network model (the NN) was established. The NN model is used with measurements of only surface temperature T_S . The NN was validated and compared with four other published global T_m models. The results show that the NN performed better than any of the four compared models on the global scale.

Prediction of hearing loss among the noise-exposed workers in a steel factory using artificial intelligence approach.

PubMed

Aliabadi, Mohsen; Farhadian, Maryam; Darvishi, Ebrahim

2015-08-01

Prediction of hearing loss in noisy workplaces is considered to be an important aspect of hearing conservation program. Artificial intelligence, as a new approach, can be used to predict the complex phenomenon such as hearing loss. Using artificial neural networks, this study aims to present an empirical model for the prediction of the hearing loss threshold among noise-exposed workers. Two hundred and ten workers employed in a steel factory were chosen, and their occupational exposure histories were collected. To determine the hearing loss threshold, the audiometric test was carried out using a calibrated audiometer. The personal noise exposure was also measured using a noise dosimeter in the workstations of workers. Finally, data obtained five variables, which can influence the hearing loss, were used for the development of the prediction model. Multilayer feed-forward neural networks with different structures were developed using MATLAB software. Neural network structures had one hidden layer with the number of neurons being approximately between 5 and 15 neurons. The best developed neural networks with one hidden layer and ten neurons could accurately predict the hearing loss threshold with RMSE = 2.6 dB and R(2) = 0.89. The results also confirmed that neural networks could provide more accurate predictions than multiple regressions. Since occupational hearing loss is frequently non-curable, results of accurate prediction can be used by occupational health experts to modify and improve noise exposure conditions.

Coordinated three-dimensional motion of the head and torso by dynamic neural networks.

PubMed

Kim, J; Hemami, H

1998-01-01

The problem of trajectory tracking control of a three dimensional (3D) model of the human upper torso and head is considered. The torso and the head are modeled as two rigid bodies connected at one point, and the Newton-Euler method is used to derive the nonlinear differential equations that govern the motion of the system. The two-link system is driven by six pairs of muscle like actuators that possess physiologically inspired alpha like and gamma like inputs, and spindle like and Golgi tendon organ like outputs. These outputs are utilized as reflex feedback for stability and stiffness control, in a long loop feedback for the purpose of estimating the state of the system (somesthesis), and as part of the input to the controller. Ideal delays of different duration are included in the feedforward and feedback paths of the system to emulate such delays encountered in physiological systems. Dynamical neural networks are trained to learn effective control of the desired maneuvers of the system. The feasibility of the controller is demonstrated by computer simulation of the successful execution of the desired maneuvers. This work demonstrates the capabilities of neural circuits in controlling highly nonlinear systems with multidelays in their feedforward and feedback paths. The ultimate long range goal of this research is toward understanding the working of the central nervous system in controlling movement. It is an interdisciplinary effort relying on mechanics, biomechanics, neuroscience, system theory, physiology and anatomy, and its short range relevance to rehabilitation must be noted.

Hermite Functional Link Neural Network for Solving the Van der Pol-Duffing Oscillator Equation.

PubMed

Mall, Susmita; Chakraverty, S

2016-08-01

Hermite polynomial-based functional link artificial neural network (FLANN) is proposed here to solve the Van der Pol-Duffing oscillator equation. A single-layer hermite neural network (HeNN) model is used, where a hidden layer is replaced by expansion block of input pattern using Hermite orthogonal polynomials. A feedforward neural network model with the unsupervised error backpropagation principle is used for modifying the network parameters and minimizing the computed error function. The Van der Pol-Duffing and Duffing oscillator equations may not be solved exactly. Here, approximate solutions of these types of equations have been obtained by applying the HeNN model for the first time. Three mathematical example problems and two real-life application problems of Van der Pol-Duffing oscillator equation, extracting the features of early mechanical failure signal and weak signal detection problems, are solved using the proposed HeNN method. HeNN approximate solutions have been compared with results obtained by the well known Runge-Kutta method. Computed results are depicted in term of graphs. After training the HeNN model, we may use it as a black box to get numerical results at any arbitrary point in the domain. Thus, the proposed HeNN method is efficient. The results reveal that this method is reliable and can be applied to other nonlinear problems too.

Laser Fluence Recognition Using Computationally Intelligent Pulsed Photoacoustics Within the Trace Gases Analysis

NASA Astrophysics Data System (ADS)

Lukić, M.; Ćojbašić, Ž.; Rabasović, M. D.; Markushev, D. D.; Todorović, D. M.

2017-11-01

In this paper, the possibilities of computational intelligence applications for trace gas monitoring are discussed. For this, pulsed infrared photoacoustics is used to investigate SF6-Ar mixtures in a multiphoton regime, assisted by artificial neural networks. Feedforward multilayer perceptron networks are applied in order to recognize both the spatial characteristics of the laser beam and the values of laser fluence Φ from the given photoacoustic signal and prevent changes. Neural networks are trained in an offline batch training regime to simultaneously estimate four parameters from theoretical or experimental photoacoustic signals: the laser beam spatial profile R(r), vibrational-to-translational relaxation time τ _{V-T} , distance from the laser beam to the absorption molecules in the photoacoustic cell r* and laser fluence Φ . The results presented in this paper show that neural networks can estimate an unknown laser beam spatial profile and the parameters of photoacoustic signals in real time and with high precision. Real-time operation, high accuracy and the possibility of application for higher intensities of radiation for a wide range of laser fluencies are factors that classify the computational intelligence approach as efficient and powerful for the in situ measurement of atmospheric pollutants.

Neural network approximation of tip-abrasion effects in AFM imaging

NASA Astrophysics Data System (ADS)

Bakucz, Peter; Yacoot, Andrew; Dziomba, Thorsten; Koenders, Ludger; Krüger-Sehm, Rolf

2008-06-01

The abrasion (wear) of tips used in scanning force microscopy (SFM) directly influences SFM image quality and is therefore of great relevance to quantitative SFM measurements. The increasing implementation of automated SFM measurement schemes has become a strong driving force for increasing efforts towards the prediction of tip wear, as it needs to be ensured that the probe is exchanged before a level of tip wear is reached that adversely affects the measurement quality. In this paper, we describe the identification of tip abrasion in a system of SFM measurements. We attempt to model the tip-abrasion process as a concatenation of a mapping from the measured AFM data to a regression vector and a nonlinear mapping from the regressor space to the output space. The mapping is formed as a basis function expansion. Feedforward neural networks are used to approximate this mapping. The one-hidden layer network gave a good quality of fit for the training and test sets for the tip-abrasion system. We illustrate our method with AFM measurements of both fine periodic structures and randomly oriented sharp features and compare our neural network results with those obtained using other methods.

ISE-based sensor array system for classification of foodstuffs

NASA Astrophysics Data System (ADS)

Ciosek, Patrycja; Sobanski, Tomasz; Augustyniak, Ewa; Wróblewski, Wojciech

2006-01-01

A system composed of an array of polymeric membrane ion-selective electrodes and a pattern recognition block—a so-called 'electronic tongue'—was used for the classification of liquid samples: milk, fruit juice and tonic. The task of this system was to automatically recognize a brand of the product. To analyze the measurement set-up responses various non-parametric classifiers such as k-nearest neighbours, a feedforward neural network and a probabilistic neural network were used. In order to enhance the classification ability of the system, standard model solutions of salts were measured (in order to take into account any variation in time of the working parameters of the sensors). This system was capable of recognizing the brand of the products with accuracy ranging from 68% to 100% (in the case of the best classifier).

Optimization of Artificial Neural Network using Evolutionary Programming for Prediction of Cascading Collapse Occurrence due to the Hidden Failure Effect

NASA Astrophysics Data System (ADS)

Idris, N. H.; Salim, N. A.; Othman, M. M.; Yasin, Z. M.

2018-03-01

This paper presents the Evolutionary Programming (EP) which proposed to optimize the training parameters for Artificial Neural Network (ANN) in predicting cascading collapse occurrence due to the effect of protection system hidden failure. The data has been collected from the probability of hidden failure model simulation from the historical data. The training parameters of multilayer-feedforward with backpropagation has been optimized with objective function to minimize the Mean Square Error (MSE). The optimal training parameters consists of the momentum rate, learning rate and number of neurons in first hidden layer and second hidden layer is selected in EP-ANN. The IEEE 14 bus system has been tested as a case study to validate the propose technique. The results show the reliable prediction of performance validated through MSE and Correlation Coefficient (R).

Retinal Connectomics: Towards Complete, Accurate Networks

PubMed Central

Marc, Robert E.; Jones, Bryan W.; Watt, Carl B.; Anderson, James R.; Sigulinsky, Crystal; Lauritzen, Scott

2013-01-01

Connectomics is a strategy for mapping complex neural networks based on high-speed automated electron optical imaging, computational assembly of neural data volumes, web-based navigational tools to explore 1012–1015 byte (terabyte to petabyte) image volumes, and annotation and markup tools to convert images into rich networks with cellular metadata. These collections of network data and associated metadata, analyzed using tools from graph theory and classification theory, can be merged with classical systems theory, giving a more completely parameterized view of how biologic information processing systems are implemented in retina and brain. Networks have two separable features: topology and connection attributes. The first findings from connectomics strongly validate the idea that the topologies complete retinal networks are far more complex than the simple schematics that emerged from classical anatomy. In particular, connectomics has permitted an aggressive refactoring of the retinal inner plexiform layer, demonstrating that network function cannot be simply inferred from stratification; exposing the complex geometric rules for inserting different cells into a shared network; revealing unexpected bidirectional signaling pathways between mammalian rod and cone systems; documenting selective feedforward systems, novel candidate signaling architectures, new coupling motifs, and the highly complex architecture of the mammalian AII amacrine cell. This is but the beginning, as the underlying principles of connectomics are readily transferrable to non-neural cell complexes and provide new contexts for assessing intercellular communication. PMID:24016532

Form and function in gene regulatory networks: the structure of network motifs determines fundamental properties of their dynamical state space.

PubMed

Ahnert, S E; Fink, T M A

2016-07-01

Network motifs have been studied extensively over the past decade, and certain motifs, such as the feed-forward loop, play an important role in regulatory networks. Recent studies have used Boolean network motifs to explore the link between form and function in gene regulatory networks and have found that the structure of a motif does not strongly determine its function, if this is defined in terms of the gene expression patterns the motif can produce. Here, we offer a different, higher-level definition of the 'function' of a motif, in terms of two fundamental properties of its dynamical state space as a Boolean network. One is the basin entropy, which is a complexity measure of the dynamics of Boolean networks. The other is the diversity of cyclic attractor lengths that a given motif can produce. Using these two measures, we examine all 104 topologically distinct three-node motifs and show that the structural properties of a motif, such as the presence of feedback loops and feed-forward loops, predict fundamental characteristics of its dynamical state space, which in turn determine aspects of its functional versatility. We also show that these higher-level properties have a direct bearing on real regulatory networks, as both basin entropy and cycle length diversity show a close correspondence with the prevalence, in neural and genetic regulatory networks, of the 13 connected motifs without self-interactions that have been studied extensively in the literature. © 2016 The Authors.

Hybrid neural network for density limit disruption prediction and avoidance on J-TEXT tokamak

NASA Astrophysics Data System (ADS)

Zheng, W.; Hu, F. R.; Zhang, M.; Chen, Z. Y.; Zhao, X. Q.; Wang, X. L.; Shi, P.; Zhang, X. L.; Zhang, X. Q.; Zhou, Y. N.; Wei, Y. N.; Pan, Y.; J-TEXT team

2018-05-01

Increasing the plasma density is one of the key methods in achieving an efficient fusion reaction. High-density operation is one of the hot topics in tokamak plasmas. Density limit disruptions remain an important issue for safe operation. An effective density limit disruption prediction and avoidance system is the key to avoid density limit disruptions for long pulse steady state operations. An artificial neural network has been developed for the prediction of density limit disruptions on the J-TEXT tokamak. The neural network has been improved from a simple multi-layer design to a hybrid two-stage structure. The first stage is a custom network which uses time series diagnostics as inputs to predict plasma density, and the second stage is a three-layer feedforward neural network to predict the probability of density limit disruptions. It is found that hybrid neural network structure, combined with radiation profile information as an input can significantly improve the prediction performance, especially the average warning time ({{T}warn} ). In particular, the {{T}warn} is eight times better than that in previous work (Wang et al 2016 Plasma Phys. Control. Fusion 58 055014) (from 5 ms to 40 ms). The success rate for density limit disruptive shots is above 90%, while, the false alarm rate for other shots is below 10%. Based on the density limit disruption prediction system and the real-time density feedback control system, the on-line density limit disruption avoidance system has been implemented on the J-TEXT tokamak.

Dynamic Analysis of Feedforward Neural Networks Using Simulated and Measured Data

DTIC Science & Technology

1988-12-01

deffiwa Ioopk(ON) fcWuk-O-k<ON-.k++) #ckflaa Ioopij(ONE.TVO) for(i0:i<ONE~i++)forj=0j<TWOI++) #"nolu HARDON RED 40 #*c4n INDETERM ITABLE/2) Pdc4nue...kiON E;i++ )forjOj<TWOj++) #frlci.a HARDON RED #4ileu INDETERN G1REEN #define HARD..OFF BLUE #if FALSE #deflne TRUE I #diflhaue FALSE 0 40 #4144h6 Outpu

Space-time adaptive decision feedback neural receivers with data selection for high-data-rate users in DS-CDMA systems.

PubMed

de Lamare, Rodrigo C; Sampaio-Neto, Raimundo

2008-11-01

A space-time adaptive decision feedback (DF) receiver using recurrent neural networks (RNNs) is proposed for joint equalization and interference suppression in direct-sequence code-division multiple-access (DS-CDMA) systems equipped with antenna arrays. The proposed receiver structure employs dynamically driven RNNs in the feedforward section for equalization and multiaccess interference (MAI) suppression and a finite impulse response (FIR) linear filter in the feedback section for performing interference cancellation. A data selective gradient algorithm, based upon the set-membership (SM) design framework, is proposed for the estimation of the coefficients of RNN structures and is applied to the estimation of the parameters of the proposed neural receiver structure. Simulation results show that the proposed techniques achieve significant performance gains over existing schemes.

Neural Generalized Predictive Control: A Newton-Raphson Implementation

NASA Technical Reports Server (NTRS)

Soloway, Donald; Haley, Pamela J.

1997-01-01

An efficient implementation of Generalized Predictive Control using a multi-layer feedforward neural network as the plant's nonlinear model is presented. In using Newton-Raphson as the optimization algorithm, the number of iterations needed for convergence is significantly reduced from other techniques. The main cost of the Newton-Raphson algorithm is in the calculation of the Hessian, but even with this overhead the low iteration numbers make Newton-Raphson faster than other techniques and a viable algorithm for real-time control. This paper presents a detailed derivation of the Neural Generalized Predictive Control algorithm with Newton-Raphson as the minimization algorithm. Simulation results show convergence to a good solution within two iterations and timing data show that real-time control is possible. Comments about the algorithm's implementation are also included.

Propagating synchrony in feed-forward networks

PubMed Central

Jahnke, Sven; Memmesheimer, Raoul-Martin; Timme, Marc

2013-01-01

Coordinated patterns of precisely timed action potentials (spikes) emerge in a variety of neural circuits but their dynamical origin is still not well understood. One hypothesis states that synchronous activity propagating through feed-forward chains of groups of neurons (synfire chains) may dynamically generate such spike patterns. Additionally, synfire chains offer the possibility to enable reliable signal transmission. So far, mostly densely connected chains, often with all-to-all connectivity between groups, have been theoretically and computationally studied. Yet, such prominent feed-forward structures have not been observed experimentally. Here we analytically and numerically investigate under which conditions diluted feed-forward chains may exhibit synchrony propagation. In addition to conventional linear input summation, we study the impact of non-linear, non-additive summation accounting for the effect of fast dendritic spikes. The non-linearities promote synchronous inputs to generate precisely timed spikes. We identify how non-additive coupling relaxes the conditions on connectivity such that it enables synchrony propagation at connectivities substantially lower than required for linearly coupled chains. Although the analytical treatment is based on a simple leaky integrate-and-fire neuron model, we show how to generalize our methods to biologically more detailed neuron models and verify our results by numerical simulations with, e.g., Hodgkin Huxley type neurons. PMID:24298251

A computational feedforward model predicts categorization of masked emotional body language for longer, but not for shorter, latencies.

PubMed

Stienen, Bernard M C; Schindler, Konrad; de Gelder, Beatrice

2012-07-01

Given the presence of massive feedback loops in brain networks, it is difficult to disentangle the contribution of feedforward and feedback processing to the recognition of visual stimuli, in this case, of emotional body expressions. The aim of the work presented in this letter is to shed light on how well feedforward processing explains rapid categorization of this important class of stimuli. By means of parametric masking, it may be possible to control the contribution of feedback activity in human participants. A close comparison is presented between human recognition performance and the performance of a computational neural model that exclusively modeled feedforward processing and was engineered to fulfill the computational requirements of recognition. Results show that the longer the stimulus onset asynchrony (SOA), the closer the performance of the human participants was to the values predicted by the model, with an optimum at an SOA of 100 ms. At short SOA latencies, human performance deteriorated, but the categorization of the emotional expressions was still above baseline. The data suggest that, although theoretically, feedback arising from inferotemporal cortex is likely to be blocked when the SOA is 100 ms, human participants still seem to rely on more local visual feedback processing to equal the model's performance.

Stochastic resonance in feedforward acupuncture networks

NASA Astrophysics Data System (ADS)

Qin, Ying-Mei; Wang, Jiang; Men, Cong; Deng, Bin; Wei, Xi-Le; Yu, Hai-Tao; Chan, Wai-Lok

2014-10-01

Effects of noises and some other network properties on the weak signal propagation are studied systematically in feedforward acupuncture networks (FFN) based on FitzHugh-Nagumo neuron model. It is found that noises with medium intensity can enhance signal propagation and this effect can be further increased by the feedforward network structure. Resonant properties in the noisy network can also be altered by several network parameters, such as heterogeneity, synapse features, and feedback connections. These results may also provide a novel potential explanation for the propagation of acupuncture signal.

New control strategies for neuroprosthetic systems.

PubMed

Crago, P E; Lan, N; Veltink, P H; Abbas, J J; Kantor, C

1996-04-01

The availability of techniques to artificially excite paralyzed muscles opens enormous potential for restoring both upper and lower extremity movements with neuroprostheses. Neuroprostheses must stimulate muscle, and control and regulate the artificial movements produced. Control methods to accomplish these tasks include feedforward (open-loop), feedback, and adaptive control. Feedforward control requires a great deal of information about the biomechanical behavior of the limb. For the upper extremity, an artificial motor program was developed to provide such movement program input to a neuroprosthesis. In lower extremity control, one group achieved their best results by attempting to meet naturally perceived gait objectives rather than to follow an exact joint angle trajectory. Adaptive feedforward control, as implemented in the cycle-to-cycle controller, gave good compensation for the gradual decrease in performance observed with open-loop control. A neural network controller was able to control its system to customize stimulation parameters in order to generate a desired output trajectory in a given individual and to maintain tracking performance in the presence of muscle fatigue. The authors believe that practical FNS control systems must exhibit many of these features of neurophysiological systems.

Jordan recurrent neural network versus IHACRES in modelling daily streamflows

NASA Astrophysics Data System (ADS)

Carcano, Elena Carla; Bartolini, Paolo; Muselli, Marco; Piroddi, Luigi

2008-12-01

SummaryA study of possible scenarios for modelling streamflow data from daily time series, using artificial neural networks (ANNs), is presented. Particular emphasis is devoted to the reconstruction of drought periods where water resource management and control are most critical. This paper considers two connectionist models: a feedforward multilayer perceptron (MLP) and a Jordan recurrent neural network (JNN), comparing network performance on real world data from two small catchments (192 and 69 km 2 in size) with irregular and torrential regimes. Several network configurations are tested to ensure a good combination of input features (rainfall and previous streamflow data) that capture the variability of the physical processes at work. Tapped delayed line (TDL) and memory effect techniques are introduced to recognize and reproduce temporal dependence. Results show a poor agreement when using TDL only, but a remarkable improvement can be obtained with JNN and its memory effect procedures, which are able to reproduce the system memory over a catchment in a more effective way. Furthermore, the IHACRES conceptual model, which relies on both rainfall and temperature input data, is introduced for comparative study. The results suggest that when good input data is unavailable, metric models perform better than conceptual ones and, in general, it is difficult to justify substantial conceptualization of complex processes.

Nonlinear identification using a B-spline neural network and chaotic immune approaches

NASA Astrophysics Data System (ADS)

dos Santos Coelho, Leandro; Pessôa, Marcelo Wicthoff

2009-11-01

One of the important applications of B-spline neural network (BSNN) is to approximate nonlinear functions defined on a compact subset of a Euclidean space in a highly parallel manner. Recently, BSNN, a type of basis function neural network, has received increasing attention and has been applied in the field of nonlinear identification. BSNNs have the potential to "learn" the process model from input-output data or "learn" fault knowledge from past experience. BSNN can be used as function approximators to construct the analytical model for residual generation too. However, BSNN is trained by gradient-based methods that may fall into local minima during the learning procedure. When using feed-forward BSNNs, the quality of approximation depends on the control points (knots) placement of spline functions. This paper describes the application of a modified artificial immune network inspired optimization method - the opt-aiNet - combined with sequences generate by Hénon map to provide a stochastic search to adjust the control points of a BSNN. The numerical results presented here indicate that artificial immune network optimization methods are useful for building good BSNN model for the nonlinear identification of two case studies: (i) the benchmark of Box and Jenkins gas furnace, and (ii) an experimental ball-and-tube system.

A classifier neural network for rotordynamic systems

NASA Astrophysics Data System (ADS)

Ganesan, R.; Jionghua, Jin; Sankar, T. S.

1995-07-01

A feedforward backpropagation neural network is formed to identify the stability characteristic of a high speed rotordynamic system. The principal focus resides in accounting for the instability due to the bearing clearance effects. The abnormal operating condition of 'normal-loose' Coulomb rub, that arises in units supported by hydrodynamic bearings or rolling element bearings, is analysed in detail. The multiple-parameter stability problem is formulated and converted to a set of three-parameter algebraic inequality equations. These three parameters map the wider range of physical parameters of commonly-used rotordynamic systems into a narrow closed region, that is used in the supervised learning of the neural network. A binary-type state of the system is expressed through these inequalities that are deduced from the analytical simulation of the rotor system. Both the hidden layer as well as functional-link networks are formed and the superiority of the functional-link network is established. Considering the real time interpretation and control of the rotordynamic system, the network reliability and the learning time are used as the evaluation criteria to assess the superiority of the functional-link network. This functional-link network is further trained using the parameter values of selected rotor systems, and the classifier network is formed. The success rate of stability status identification is obtained to assess the potentials of this classifier network. The classifier network is shown that it can also be used, for control purposes, as an 'advisory' system that suggests the optimum way of parameter adjustment.

Application of neural models as controllers in mobile robot velocity control loop

NASA Astrophysics Data System (ADS)

Cerkala, Jakub; Jadlovska, Anna

2017-01-01

This paper presents the application of an inverse neural models used as controllers in comparison to classical PI controllers for velocity tracking control task used in two-wheel, differentially driven mobile robot. The PI controller synthesis is based on linear approximation of actuators with equivalent load. In order to obtain relevant datasets for training of feed-forward multi-layer perceptron based neural network used as neural model, the mathematical model of mobile robot, that combines its kinematic and dynamic properties such as chassis dimensions, center of gravity offset, friction and actuator parameters is used. Neural models are trained off-line to act as an inverse dynamics of DC motors with particular load using data collected in simulation experiment for motor input voltage step changes within bounded operating area. The performances of PI controllers versus inverse neural models in mobile robot internal velocity control loops are demonstrated and compared in simulation experiment of navigation control task for line segment motion in plane.

Prediction of Weld Penetration in FCAW of HSLA steel using Artificial Neural Networks

NASA Astrophysics Data System (ADS)

Asl, Y. Dadgar; Mostafa, N. B.; Panahizadeh R., V.; Seyedkashi, S. M. H.

2011-01-01

Flux-cored arc welding (FCAW) is a semiautomatic or automatic arc welding process that requires a continuously-fed consumable tubular electrode containing a flux. The main FCAW process parameters affecting the depth of penetration are welding current, arc voltage, nozzle-to-work distance, torch angle and welding speed. Shallow depth of penetration may contribute to failure of a welded structure since penetration determines the stress-carrying capacity of a welded joint. To avoid such occurrences; the welding process parameters influencing the weld penetration must be properly selected to obtain an acceptable weld penetration and hence a high quality joint. Artificial neural networks (ANN), also called neural networks (NN), are computational models used to express complex non-linear relationships between input and output data. In this paper, artificial neural network (ANN) method is used to predict the effects of welding current, arc voltage, nozzle-to-work distance, torch angle and welding speed on weld penetration depth in gas shielded FCAW of a grade of high strength low alloy steel. 32 experimental runs were carried out using the bead-on-plate welding technique. Weld penetrations were measured and on the basis of these 32 sets of experimental data, a feed-forward back-propagation neural network was created. 28 sets of the experiments were used as the training data and the remaining 4 sets were used for the testing phase of the network. The ANN has one hidden layer with eight neurons and is trained after 840 iterations. The comparison between the experimental results and ANN results showed that the trained network could predict the effects of the FCAW process parameters on weld penetration adequately.

The Prediction of the Risk Level of Pulmonary Embolism and Deep Vein Thrombosis through Artificial Neural Network

PubMed Central

Agharezaei, Laleh; Agharezaei, Zhila; Nemati, Ali; Bahaadinbeigy, Kambiz; Keynia, Farshid; Baneshi, Mohammad Reza; Iranpour, Abedin; Agharezaei, Moslem

2016-01-01

Background: Venous thromboembolism is a common cause of mortality among hospitalized patients and yet it is preventable through detecting the precipitating factors and a prompt diagnosis by specialists. The present study has been carried out in order to assist specialists in the diagnosis and prediction of the risk level of pulmonary embolism in patients, by means of artificial neural network. Method: A number of 31 risk factors have been used in this study in order to evaluate the conditions of 294 patients hospitalized in 3 educational hospitals affiliated with Kerman University of Medical Sciences. Two types of artificial neural networks, namely Feed-Forward Back Propagation and Elman Back Propagation, were compared in this study. Results: Through an optimized artificial neural network model, an accuracy and risk level index of 93.23 percent was achieved and, subsequently, the results have been compared with those obtained from the perfusion scan of the patients. 86.61 percent of high risk patients diagnosed through perfusion scan diagnostic method were also diagnosed correctly through the method proposed in the present study. Conclusions: The results of this study can be a good resource for physicians, medical assistants, and healthcare staff to diagnose high risk patients more precisely and prevent the mortalities. Additionally, expenses and other unnecessary diagnostic methods such as perfusion scans can be efficiently reduced. PMID:28077893

The Prediction of the Risk Level of Pulmonary Embolism and Deep Vein Thrombosis through Artificial Neural Network.

PubMed

Agharezaei, Laleh; Agharezaei, Zhila; Nemati, Ali; Bahaadinbeigy, Kambiz; Keynia, Farshid; Baneshi, Mohammad Reza; Iranpour, Abedin; Agharezaei, Moslem

2016-10-01

Venous thromboembolism is a common cause of mortality among hospitalized patients and yet it is preventable through detecting the precipitating factors and a prompt diagnosis by specialists. The present study has been carried out in order to assist specialists in the diagnosis and prediction of the risk level of pulmonary embolism in patients, by means of artificial neural network. A number of 31 risk factors have been used in this study in order to evaluate the conditions of 294 patients hospitalized in 3 educational hospitals affiliated with Kerman University of Medical Sciences. Two types of artificial neural networks, namely Feed-Forward Back Propagation and Elman Back Propagation, were compared in this study. Through an optimized artificial neural network model, an accuracy and risk level index of 93.23 percent was achieved and, subsequently, the results have been compared with those obtained from the perfusion scan of the patients. 86.61 percent of high risk patients diagnosed through perfusion scan diagnostic method were also diagnosed correctly through the method proposed in the present study. The results of this study can be a good resource for physicians, medical assistants, and healthcare staff to diagnose high risk patients more precisely and prevent the mortalities. Additionally, expenses and other unnecessary diagnostic methods such as perfusion scans can be efficiently reduced.

A heuristic neural network initialization scheme for modeling nonlinear functions in engineering mechanics: continuous development

NASA Astrophysics Data System (ADS)

Pei, Jin-Song; Mai, Eric C.

2007-04-01

This paper introduces a continuous effort towards the development of a heuristic initialization methodology for constructing multilayer feedforward neural networks to model nonlinear functions. In this and previous studies that this work is built upon, including the one presented at SPIE 2006, the authors do not presume to provide a universal method to approximate arbitrary functions, rather the focus is given to the development of a rational and unambiguous initialization procedure that applies to the approximation of nonlinear functions in the specific domain of engineering mechanics. The applications of this exploratory work can be numerous including those associated with potential correlation and interpretation of the inner workings of neural networks, such as damage detection. The goal of this study is fulfilled by utilizing the governing physics and mathematics of nonlinear functions and the strength of the sigmoidal basis function. A step-by-step graphical procedure utilizing a few neural network prototypes as "templates" to approximate commonly seen memoryless nonlinear functions of one or two variables is further developed in this study. Decomposition of complex nonlinear functions into a summation of some simpler nonlinear functions is utilized to exploit this prototype-based initialization methodology. Training examples are presented to demonstrate the rationality and effciency of the proposed methodology when compared with the popular Nguyen-Widrow initialization algorithm. Future work is also identfied.

Supervised artificial neural network-based method for conversion of solar radiation data (case study: Algeria)

NASA Astrophysics Data System (ADS)

Laidi, Maamar; Hanini, Salah; Rezrazi, Ahmed; Yaiche, Mohamed Redha; El Hadj, Abdallah Abdallah; Chellali, Farouk

2017-04-01

In this study, a backpropagation artificial neural network (BP-ANN) model is used as an alternative approach to predict solar radiation on tilted surfaces (SRT) using a number of variables involved in physical process. These variables are namely the latitude of the site, mean temperature and relative humidity, Linke turbidity factor and Angstrom coefficient, extraterrestrial solar radiation, solar radiation data measured on horizontal surfaces (SRH), and solar zenith angle. Experimental solar radiation data from 13 stations spread all over Algeria around the year (2004) were used for training/validation and testing the artificial neural networks (ANNs), and one station was used to make the interpolation of the designed ANN. The ANN model was trained, validated, and tested using 60, 20, and 20 % of all data, respectively. The configuration 8-35-1 (8 inputs, 35 hidden, and 1 output neurons) presented an excellent agreement between the prediction and the experimental data during the test stage with determination coefficient of 0.99 and root meat squared error of 5.75 Wh/m2, considering a three-layer feedforward backpropagation neural network with Levenberg-Marquardt training algorithm, a hyperbolic tangent sigmoid and linear transfer function at the hidden and the output layer, respectively. This novel model could be used by researchers or scientists to design high-efficiency solar devices that are usually tilted at an optimum angle to increase the solar incident on the surface.

Detecting central fixation by means of artificial neural networks in a pediatric vision screener using retinal birefringence scanning.

PubMed

Gramatikov, Boris I

2017-04-27

Reliable detection of central fixation and eye alignment is essential in the diagnosis of amblyopia ("lazy eye"), which can lead to blindness. Our lab has developed and reported earlier a pediatric vision screener that performs scanning of the retina around the fovea and analyzes changes in the polarization state of light as the scan progresses. Depending on the direction of gaze and the instrument design, the screener produces several signal frequencies that can be utilized in the detection of central fixation. The objective of this study was to compare artificial neural networks with classical statistical methods, with respect to their ability to detect central fixation reliably. A classical feedforward, pattern recognition, two-layer neural network architecture was used, consisting of one hidden layer and one output layer. The network has four inputs, representing normalized spectral powers at four signal frequencies generated during retinal birefringence scanning. The hidden layer contains four neurons. The output suggests presence or absence of central fixation. Backpropagation was used to train the network, using the gradient descent algorithm and the cross-entropy error as the performance function. The network was trained, validated and tested on a set of controlled calibration data obtained from 600 measurements from ten eyes in a previous study, and was additionally tested on a clinical set of 78 eyes, independently diagnosed by an ophthalmologist. In the first part of this study, a neural network was designed around the calibration set. With a proper architecture and training, the network provided performance that was comparable to classical statistical methods, allowing perfect separation between the central and paracentral fixation data, with both the sensitivity and the specificity of the instrument being 100%. In the second part of the study, the neural network was applied to the clinical data. It allowed reliable separation between normal subjects and affected subjects, its accuracy again matching that of the statistical methods. With a proper choice of a neural network architecture and a good, uncontaminated training data set, the artificial neural network can be an efficient classification tool for detecting central fixation based on retinal birefringence scanning.

CNN universal machine as classificaton platform: an art-like clustering algorithm.

PubMed

Bálya, David

2003-12-01

Fast and robust classification of feature vectors is a crucial task in a number of real-time systems. A cellular neural/nonlinear network universal machine (CNN-UM) can be very efficient as a feature detector. The next step is to post-process the results for object recognition. This paper shows how a robust classification scheme based on adaptive resonance theory (ART) can be mapped to the CNN-UM. Moreover, this mapping is general enough to include different types of feed-forward neural networks. The designed analogic CNN algorithm is capable of classifying the extracted feature vectors keeping the advantages of the ART networks, such as robust, plastic and fault-tolerant behaviors. An analogic algorithm is presented for unsupervised classification with tunable sensitivity and automatic new class creation. The algorithm is extended for supervised classification. The presented binary feature vector classification is implemented on the existing standard CNN-UM chips for fast classification. The experimental evaluation shows promising performance after 100% accuracy on the training set.

SVM-based tree-type neural networks as a critic in adaptive critic designs for control.

PubMed

Deb, Alok Kanti; Jayadeva; Gopal, Madan; Chandra, Suresh

2007-07-01

In this paper, we use the approach of adaptive critic design (ACD) for control, specifically, the action-dependent heuristic dynamic programming (ADHDP) method. A least squares support vector machine (SVM) regressor has been used for generating the control actions, while an SVM-based tree-type neural network (NN) is used as the critic. After a failure occurs, the critic and action are retrained in tandem using the failure data. Failure data is binary classification data, where the number of failure states are very few as compared to the number of no-failure states. The difficulty of conventional multilayer feedforward NNs in learning this type of classification data has been overcome by using the SVM-based tree-type NN, which due to its feature to add neurons to learn misclassified data, has the capability to learn any binary classification data without a priori choice of the number of neurons or the structure of the network. The capability of the trained controller to handle unforeseen situations is demonstrated.

Crystal surface analysis using matrix textural features classified by a probabilistic neural network

NASA Astrophysics Data System (ADS)

Sawyer, Curry R.; Quach, Viet; Nason, Donald; van den Berg, Lodewijk

1991-12-01

A system is under development in which surface quality of a growing bulk mercuric iodide crystal is monitored by video camera at regular intervals for early detection of growth irregularities. Mercuric iodide single crystals are employed in radiation detectors. A microcomputer system is used for image capture and processing. The digitized image is divided into multiple overlapping sub-images and features are extracted from each sub-image based on statistical measures of the gray tone distribution, according to the method of Haralick. Twenty parameters are derived from each sub-image and presented to a probabilistic neural network (PNN) for classification. This number of parameters was found to be optimal for the system. The PNN is a hierarchical, feed-forward network that can be rapidly reconfigured as additional training data become available. Training data is gathered by reviewing digital images of many crystals during their growth cycle and compiling two sets of images, those with and without irregularities.

Assessing artificial neural networks and statistical methods for infilling missing soil moisture records

NASA Astrophysics Data System (ADS)

Dumedah, Gift; Walker, Jeffrey P.; Chik, Li

2014-07-01

Soil moisture information is critically important for water management operations including flood forecasting, drought monitoring, and groundwater recharge estimation. While an accurate and continuous record of soil moisture is required for these applications, the available soil moisture data, in practice, is typically fraught with missing values. There are a wide range of methods available to infilling hydrologic variables, but a thorough inter-comparison between statistical methods and artificial neural networks has not been made. This study examines 5 statistical methods including monthly averages, weighted Pearson correlation coefficient, a method based on temporal stability of soil moisture, and a weighted merging of the three methods, together with a method based on the concept of rough sets. Additionally, 9 artificial neural networks are examined, broadly categorized into feedforward, dynamic, and radial basis networks. These 14 infilling methods were used to estimate missing soil moisture records and subsequently validated against known values for 13 soil moisture monitoring stations for three different soil layer depths in the Yanco region in southeast Australia. The evaluation results show that the top three highest performing methods are the nonlinear autoregressive neural network, rough sets method, and monthly replacement. A high estimation accuracy (root mean square error (RMSE) of about 0.03 m/m) was found in the nonlinear autoregressive network, due to its regression based dynamic network which allows feedback connections through discrete-time estimation. An equally high accuracy (0.05 m/m RMSE) in the rough sets procedure illustrates the important role of temporal persistence of soil moisture, with the capability to account for different soil moisture conditions.

A recurrent self-organizing neural fuzzy inference network.

PubMed

Juang, C F; Lin, C T

1999-01-01

A recurrent self-organizing neural fuzzy inference network (RSONFIN) is proposed in this paper. The RSONFIN is inherently a recurrent multilayered connectionist network for realizing the basic elements and functions of dynamic fuzzy inference, and may be considered to be constructed from a series of dynamic fuzzy rules. The temporal relations embedded in the network are built by adding some feedback connections representing the memory elements to a feedforward neural fuzzy network. Each weight as well as node in the RSONFIN has its own meaning and represents a special element in a fuzzy rule. There are no hidden nodes (i.e., no membership functions and fuzzy rules) initially in the RSONFIN. They are created on-line via concurrent structure identification (the construction of dynamic fuzzy if-then rules) and parameter identification (the tuning of the free parameters of membership functions). The structure learning together with the parameter learning forms a fast learning algorithm for building a small, yet powerful, dynamic neural fuzzy network. Two major characteristics of the RSONFIN can thus be seen: 1) the recurrent property of the RSONFIN makes it suitable for dealing with temporal problems and 2) no predetermination, like the number of hidden nodes, must be given, since the RSONFIN can find its optimal structure and parameters automatically and quickly. Moreover, to reduce the number of fuzzy rules generated, a flexible input partition method, the aligned clustering-based algorithm, is proposed. Various simulations on temporal problems are done and performance comparisons with some existing recurrent networks are also made. Efficiency of the RSONFIN is verified from these results.

Task-specific feature extraction and classification of fMRI volumes using a deep neural network initialized with a deep belief network: Evaluation using sensorimotor tasks

PubMed Central

Jang, Hojin; Plis, Sergey M.; Calhoun, Vince D.; Lee, Jong-Hwan

2016-01-01

Feedforward deep neural networks (DNN), artificial neural networks with multiple hidden layers, have recently demonstrated a record-breaking performance in multiple areas of applications in computer vision and speech processing. Following the success, DNNs have been applied to neuroimaging modalities including functional/structural magnetic resonance imaging (MRI) and positron-emission tomography data. However, no study has explicitly applied DNNs to 3D whole-brain fMRI volumes and thereby extracted hidden volumetric representations of fMRI that are discriminative for a task performed as the fMRI volume was acquired. Our study applied fully connected feedforward DNN to fMRI volumes collected in four sensorimotor tasks (i.e., left-hand clenching, right-hand clenching, auditory attention, and visual stimulus) undertaken by 12 healthy participants. Using a leave-one-subject-out cross-validation scheme, a restricted Boltzmann machine-based deep belief network was pretrained and used to initialize weights of the DNN. The pretrained DNN was fine-tuned while systematically controlling weight-sparsity levels across hidden layers. Optimal weight-sparsity levels were determined from a minimum validation error rate of fMRI volume classification. Minimum error rates (mean ± standard deviation; %) of 6.9 (± 3.8) were obtained from the three-layer DNN with the sparsest condition of weights across the three hidden layers. These error rates were even lower than the error rates from the single-layer network (9.4 ± 4.6) and the two-layer network (7.4 ± 4.1). The estimated DNN weights showed spatial patterns that are remarkably task-specific, particularly in the higher layers. The output values of the third hidden layer represented distinct patterns/codes of the 3D whole-brain fMRI volume and encoded the information of the tasks as evaluated from representational similarity analysis. Our reported findings show the ability of the DNN to classify a single fMRI volume based on the extraction of hidden representations of fMRI volumes associated with tasks across multiple hidden layers. Our study may be beneficial to the automatic classification/diagnosis of neuropsychiatric and neurological diseases and prediction of disease severity and recovery in (pre-) clinical settings using fMRI volumes without requiring an estimation of activation patterns or ad hoc statistical evaluation. PMID:27079534

Task-specific feature extraction and classification of fMRI volumes using a deep neural network initialized with a deep belief network: Evaluation using sensorimotor tasks.

PubMed

Jang, Hojin; Plis, Sergey M; Calhoun, Vince D; Lee, Jong-Hwan

2017-01-15

Feedforward deep neural networks (DNNs), artificial neural networks with multiple hidden layers, have recently demonstrated a record-breaking performance in multiple areas of applications in computer vision and speech processing. Following the success, DNNs have been applied to neuroimaging modalities including functional/structural magnetic resonance imaging (MRI) and positron-emission tomography data. However, no study has explicitly applied DNNs to 3D whole-brain fMRI volumes and thereby extracted hidden volumetric representations of fMRI that are discriminative for a task performed as the fMRI volume was acquired. Our study applied fully connected feedforward DNN to fMRI volumes collected in four sensorimotor tasks (i.e., left-hand clenching, right-hand clenching, auditory attention, and visual stimulus) undertaken by 12 healthy participants. Using a leave-one-subject-out cross-validation scheme, a restricted Boltzmann machine-based deep belief network was pretrained and used to initialize weights of the DNN. The pretrained DNN was fine-tuned while systematically controlling weight-sparsity levels across hidden layers. Optimal weight-sparsity levels were determined from a minimum validation error rate of fMRI volume classification. Minimum error rates (mean±standard deviation; %) of 6.9 (±3.8) were obtained from the three-layer DNN with the sparsest condition of weights across the three hidden layers. These error rates were even lower than the error rates from the single-layer network (9.4±4.6) and the two-layer network (7.4±4.1). The estimated DNN weights showed spatial patterns that are remarkably task-specific, particularly in the higher layers. The output values of the third hidden layer represented distinct patterns/codes of the 3D whole-brain fMRI volume and encoded the information of the tasks as evaluated from representational similarity analysis. Our reported findings show the ability of the DNN to classify a single fMRI volume based on the extraction of hidden representations of fMRI volumes associated with tasks across multiple hidden layers. Our study may be beneficial to the automatic classification/diagnosis of neuropsychiatric and neurological diseases and prediction of disease severity and recovery in (pre-) clinical settings using fMRI volumes without requiring an estimation of activation patterns or ad hoc statistical evaluation. Copyright © 2016 Elsevier Inc. All rights reserved.

Optimization of Adaboost Algorithm for Sonar Target Detection in a Multi-Stage ATR System

NASA Technical Reports Server (NTRS)

Lin, Tsung Han (Hank)

2011-01-01

JPL has developed a multi-stage Automated Target Recognition (ATR) system to locate objects in images. First, input images are preprocessed and sent to a Grayscale Optical Correlator (GOC) filter to identify possible regions-of-interest (ROIs). Second, feature extraction operations are performed using Texton filters and Principal Component Analysis (PCA). Finally, the features are fed to a classifier, to identify ROIs that contain the targets. Previous work used the Feed-forward Back-propagation Neural Network for classification. In this project we investigate a version of Adaboost as a classifier for comparison. The version we used is known as GentleBoost. We used the boosted decision tree as the weak classifier. We have tested our ATR system against real-world sonar images using the Adaboost approach. Results indicate an improvement in performance over a single Neural Network design.

A comparison of neural network and fuzzy clustering techniques in segmenting magnetic resonance images of the brain.

PubMed

Hall, L O; Bensaid, A M; Clarke, L P; Velthuizen, R P; Silbiger, M S; Bezdek, J C

1992-01-01

Magnetic resonance (MR) brain section images are segmented and then synthetically colored to give visual representations of the original data with three approaches: the literal and approximate fuzzy c-means unsupervised clustering algorithms, and a supervised computational neural network. Initial clinical results are presented on normal volunteers and selected patients with brain tumors surrounded by edema. Supervised and unsupervised segmentation techniques provide broadly similar results. Unsupervised fuzzy algorithms were visually observed to show better segmentation when compared with raw image data for volunteer studies. For a more complex segmentation problem with tumor/edema or cerebrospinal fluid boundary, where the tissues have similar MR relaxation behavior, inconsistency in rating among experts was observed, with fuzz-c-means approaches being slightly preferred over feedforward cascade correlation results. Various facets of both approaches, such as supervised versus unsupervised learning, time complexity, and utility for the diagnostic process, are compared.

Least Squares Neural Network-Based Wireless E-Nose System Using an SnO₂ Sensor Array.

PubMed

Shahid, Areej; Choi, Jong-Hyeok; Rana, Abu Ul Hassan Sarwar; Kim, Hyun-Seok

2018-05-06

Over the last few decades, the development of the electronic nose (E-nose) for detection and quantification of dangerous and odorless gases, such as methane (CH₄) and carbon monoxide (CO), using an array of SnO₂ gas sensors has attracted considerable attention. This paper addresses sensor cross sensitivity by developing a classifier and estimator using an artificial neural network (ANN) and least squares regression (LSR), respectively. Initially, the ANN was implemented using a feedforward pattern recognition algorithm to learn the collective behavior of an array as the signature of a particular gas. In the second phase, the classified gas was quantified by minimizing the mean square error using LSR. The combined approach produced 98.7% recognition probability, with 95.5 and 94.4% estimated gas concentration accuracies for CH₄ and CO, respectively. The classifier and estimator parameters were deployed in a remote microcontroller for the actualization of a wireless E-nose system.

Condition monitoring of an electro-magnetic brake using an artificial neural network

NASA Astrophysics Data System (ADS)

Gofran, T.; Neugebauer, P.; Schramm, D.

2017-10-01

This paper presents a data-driven approach to Condition Monitoring of Electromagnetic brakes without use of additional sensors. For safe and efficient operation of electric motor a regular evaluation and replacement of the friction surface of the brake is required. One such evaluation method consists of direct or indirect sensing of the air-gap between pressure plate and magnet. A larger gap is generally indicative of worn surface(s). Traditionally this has been accomplished by the use of additional sensors - making existing systems complex, cost- sensitive and difficult to maintain. In this work a feed-forward Artificial Neural Network (ANN) is learned with the electrical data of the brake by supervised learning method to estimate the air-gap. The ANN model is optimized on the training set and validated using the test set. The experimental results of estimated air-gap with accuracy of over 95% demonstrate the validity of the proposed approach.

Simulative and experimental investigation on stator winding turn and unbalanced supply voltage fault diagnosis in induction motors using Artificial Neural Networks.

PubMed

Lashkari, Negin; Poshtan, Javad; Azgomi, Hamid Fekri

2015-11-01

The three-phase shift between line current and phase voltage of induction motors can be used as an efficient fault indicator to detect and locate inter-turn stator short-circuit (ITSC) fault. However, unbalanced supply voltage is one of the contributing factors that inevitably affect stator currents and therefore the three-phase shift. Thus, it is necessary to propose a method that is able to identify whether the unbalance of three currents is caused by ITSC or supply voltage fault. This paper presents a feedforward multilayer-perceptron Neural Network (NN) trained by back propagation, based on monitoring negative sequence voltage and the three-phase shift. The data which are required for training and test NN are generated using simulated model of stator. The experimental results are presented to verify the superior accuracy of the proposed method. Copyright © 2015. Published by Elsevier Ltd.

Online handwritten mathematical expression recognition

NASA Astrophysics Data System (ADS)

Büyükbayrak, Hakan; Yanikoglu, Berrin; Erçil, Aytül

2007-01-01

We describe a system for recognizing online, handwritten mathematical expressions. The system is designed with a user-interface for writing scientific articles, supporting the recognition of basic mathematical expressions as well as integrals, summations, matrices etc. A feed-forward neural network recognizes symbols which are assumed to be single-stroke and a recursive algorithm parses the expression by combining neural network output and the structure of the expression. Preliminary results show that writer-dependent recognition rates are very high (99.8%) while writer-independent symbol recognition rates are lower (75%). The interface associated with the proposed system integrates the built-in recognition capabilities of the Microsoft's Tablet PC API for recognizing textual input and supports conversion of hand-drawn figures into PNG format. This enables the user to enter text, mathematics and draw figures in a single interface. After recognition, all output is combined into one LATEX code and compiled into a PDF file.

Cascade control of superheated steam temperature with neuro-PID controller.

PubMed

Zhang, Jianhua; Zhang, Fenfang; Ren, Mifeng; Hou, Guolian; Fang, Fang

2012-11-01

In this paper, an improved cascade control methodology for superheated processes is developed, in which the primary PID controller is implemented by neural networks trained by minimizing error entropy criterion. The entropy of the tracking error can be estimated recursively by utilizing receding horizon window technique. The measurable disturbances in superheated processes are input to the neuro-PID controller besides the sequences of tracking error in outer loop control system, hence, feedback control is combined with feedforward control in the proposed neuro-PID controller. The convergent condition of the neural networks is analyzed. The implementation procedures of the proposed cascade control approach are summarized. Compared with the neuro-PID controller using minimizing squared error criterion, the proposed neuro-PID controller using minimizing error entropy criterion may decrease fluctuations of the superheated steam temperature. A simulation example shows the advantages of the proposed method. Copyright © 2012 ISA. Published by Elsevier Ltd. All rights reserved.

GA-based fuzzy reinforcement learning for control of a magnetic bearing system.

PubMed

Lin, C T; Jou, C P

2000-01-01

This paper proposes a TD (temporal difference) and GA (genetic algorithm)-based reinforcement (TDGAR) learning method and applies it to the control of a real magnetic bearing system. The TDGAR learning scheme is a new hybrid GA, which integrates the TD prediction method and the GA to perform the reinforcement learning task. The TDGAR learning system is composed of two integrated feedforward networks. One neural network acts as a critic network to guide the learning of the other network (the action network) which determines the outputs (actions) of the TDGAR learning system. The action network can be a normal neural network or a neural fuzzy network. Using the TD prediction method, the critic network can predict the external reinforcement signal and provide a more informative internal reinforcement signal to the action network. The action network uses the GA to adapt itself according to the internal reinforcement signal. The key concept of the TDGAR learning scheme is to formulate the internal reinforcement signal as the fitness function for the GA such that the GA can evaluate the candidate solutions (chromosomes) regularly, even during periods without external feedback from the environment. This enables the GA to proceed to new generations regularly without waiting for the arrival of the external reinforcement signal. This can usually accelerate the GA learning since a reinforcement signal may only be available at a time long after a sequence of actions has occurred in the reinforcement learning problem. The proposed TDGAR learning system has been used to control an active magnetic bearing (AMB) system in practice. A systematic design procedure is developed to achieve successful integration of all the subsystems including magnetic suspension, mechanical structure, and controller training. The results show that the TDGAR learning scheme can successfully find a neural controller or a neural fuzzy controller for a self-designed magnetic bearing system.

Reinforcement Learning of Linking and Tracing Contours in Recurrent Neural Networks

PubMed Central

Brosch, Tobias; Neumann, Heiko; Roelfsema, Pieter R.

2015-01-01

The processing of a visual stimulus can be subdivided into a number of stages. Upon stimulus presentation there is an early phase of feedforward processing where the visual information is propagated from lower to higher visual areas for the extraction of basic and complex stimulus features. This is followed by a later phase where horizontal connections within areas and feedback connections from higher areas back to lower areas come into play. In this later phase, image elements that are behaviorally relevant are grouped by Gestalt grouping rules and are labeled in the cortex with enhanced neuronal activity (object-based attention in psychology). Recent neurophysiological studies revealed that reward-based learning influences these recurrent grouping processes, but it is not well understood how rewards train recurrent circuits for perceptual organization. This paper examines the mechanisms for reward-based learning of new grouping rules. We derive a learning rule that can explain how rewards influence the information flow through feedforward, horizontal and feedback connections. We illustrate the efficiency with two tasks that have been used to study the neuronal correlates of perceptual organization in early visual cortex. The first task is called contour-integration and demands the integration of collinear contour elements into an elongated curve. We show how reward-based learning causes an enhancement of the representation of the to-be-grouped elements at early levels of a recurrent neural network, just as is observed in the visual cortex of monkeys. The second task is curve-tracing where the aim is to determine the endpoint of an elongated curve composed of connected image elements. If trained with the new learning rule, neural networks learn to propagate enhanced activity over the curve, in accordance with neurophysiological data. We close the paper with a number of model predictions that can be tested in future neurophysiological and computational studies. PMID:26496502

LANGUAGE EXPERIENCE SHAPES PROCESSING OF PITCH RELEVANT INFORMATION IN THE HUMAN BRAINSTEM AND AUDITORY CORTEX: ELECTROPHYSIOLOGICAL EVIDENCE.

PubMed

Krishnan, Ananthanarayan; Gandour, Jackson T

2014-12-01

Pitch is a robust perceptual attribute that plays an important role in speech, language, and music. As such, it provides an analytic window to evaluate how neural activity relevant to pitch undergo transformation from early sensory to later cognitive stages of processing in a well coordinated hierarchical network that is subject to experience-dependent plasticity. We review recent evidence of language experience-dependent effects in pitch processing based on comparisons of native vs. nonnative speakers of a tonal language from electrophysiological recordings in the auditory brainstem and auditory cortex. We present evidence that shows enhanced representation of linguistically-relevant pitch dimensions or features at both the brainstem and cortical levels with a stimulus-dependent preferential activation of the right hemisphere in native speakers of a tone language. We argue that neural representation of pitch-relevant information in the brainstem and early sensory level processing in the auditory cortex is shaped by the perceptual salience of domain-specific features. While both stages of processing are shaped by language experience, neural representations are transformed and fundamentally different at each biological level of abstraction. The representation of pitch relevant information in the brainstem is more fine-grained spectrotemporally as it reflects sustained neural phase-locking to pitch relevant periodicities contained in the stimulus. In contrast, the cortical pitch relevant neural activity reflects primarily a series of transient temporal neural events synchronized to certain temporal attributes of the pitch contour. We argue that experience-dependent enhancement of pitch representation for Chinese listeners most likely reflects an interaction between higher-level cognitive processes and early sensory-level processing to improve representations of behaviorally-relevant features that contribute optimally to perception. It is our view that long-term experience shapes this adaptive process wherein the top-down connections provide selective gating of inputs to both cortical and subcortical structures to enhance neural responses to specific behaviorally-relevant attributes of the stimulus. A theoretical framework for a neural network is proposed involving coordination between local, feedforward, and feedback components that can account for experience-dependent enhancement of pitch representations at multiple levels of the auditory pathway. The ability to record brainstem and cortical pitch relevant responses concurrently may provide a new window to evaluate the online interplay between feedback, feedforward, and local intrinsic components in the hierarchical processing of pitch relevant information.

LANGUAGE EXPERIENCE SHAPES PROCESSING OF PITCH RELEVANT INFORMATION IN THE HUMAN BRAINSTEM AND AUDITORY CORTEX: ELECTROPHYSIOLOGICAL EVIDENCE

PubMed Central

Krishnan, Ananthanarayan; Gandour, Jackson T.

2015-01-01

Pitch is a robust perceptual attribute that plays an important role in speech, language, and music. As such, it provides an analytic window to evaluate how neural activity relevant to pitch undergo transformation from early sensory to later cognitive stages of processing in a well coordinated hierarchical network that is subject to experience-dependent plasticity. We review recent evidence of language experience-dependent effects in pitch processing based on comparisons of native vs. nonnative speakers of a tonal language from electrophysiological recordings in the auditory brainstem and auditory cortex. We present evidence that shows enhanced representation of linguistically-relevant pitch dimensions or features at both the brainstem and cortical levels with a stimulus-dependent preferential activation of the right hemisphere in native speakers of a tone language. We argue that neural representation of pitch-relevant information in the brainstem and early sensory level processing in the auditory cortex is shaped by the perceptual salience of domain-specific features. While both stages of processing are shaped by language experience, neural representations are transformed and fundamentally different at each biological level of abstraction. The representation of pitch relevant information in the brainstem is more fine-grained spectrotemporally as it reflects sustained neural phase-locking to pitch relevant periodicities contained in the stimulus. In contrast, the cortical pitch relevant neural activity reflects primarily a series of transient temporal neural events synchronized to certain temporal attributes of the pitch contour. We argue that experience-dependent enhancement of pitch representation for Chinese listeners most likely reflects an interaction between higher-level cognitive processes and early sensory-level processing to improve representations of behaviorally-relevant features that contribute optimally to perception. It is our view that long-term experience shapes this adaptive process wherein the top-down connections provide selective gating of inputs to both cortical and subcortical structures to enhance neural responses to specific behaviorally-relevant attributes of the stimulus. A theoretical framework for a neural network is proposed involving coordination between local, feedforward, and feedback components that can account for experience-dependent enhancement of pitch representations at multiple levels of the auditory pathway. The ability to record brainstem and cortical pitch relevant responses concurrently may provide a new window to evaluate the online interplay between feedback, feedforward, and local intrinsic components in the hierarchical processing of pitch relevant information. PMID:25838636

Neural dynamic programming and its application to control systems

NASA Astrophysics Data System (ADS)

Seong, Chang-Yun

There are few general practical feedback control methods for nonlinear MIMO (multi-input-multi-output) systems, although such methods exist for their linear counterparts. Neural Dynamic Programming (NDP) is proposed as a practical design method of optimal feedback controllers for nonlinear MIMO systems. NDP is an offspring of both neural networks and optimal control theory. In optimal control theory, the optimal solution to any nonlinear MIMO control problem may be obtained from the Hamilton-Jacobi-Bellman equation (HJB) or the Euler-Lagrange equations (EL). The two sets of equations provide the same solution in different forms: EL leads to a sequence of optimal control vectors, called Feedforward Optimal Control (FOC); HJB yields a nonlinear optimal feedback controller, called Dynamic Programming (DP). DP produces an optimal solution that can reject disturbances and uncertainties as a result of feedback. Unfortunately, computation and storage requirements associated with DP solutions can be problematic, especially for high-order nonlinear systems. This dissertation presents an approximate technique for solving the DP problem based on neural network techniques that provides many of the performance benefits (e.g., optimality and feedback) of DP and benefits from the numerical properties of neural networks. We formulate neural networks to approximate optimal feedback solutions whose existence DP justifies. We show the conditions under which NDP closely approximates the optimal solution. Finally, we introduce the learning operator characterizing the learning process of the neural network in searching the optimal solution. The analysis of the learning operator provides not only a fundamental understanding of the learning process in neural networks but also useful guidelines for selecting the number of weights of the neural network. As a result, NDP finds---with a reasonable amount of computation and storage---the optimal feedback solutions to nonlinear MIMO control problems that would be very difficult to solve with DP. NDP was demonstrated on several applications such as the lateral autopilot logic for a Boeing 747, the minimum fuel control of a double-integrator plant with bounded control, the backward steering of a two-trailer truck, and the set-point control of a two-link robot arm.

Prediction Analysis of Weld-Bead and Heat Affected Zone in TIG welding using Artificial Neural Networks

NASA Astrophysics Data System (ADS)

Saldanha, Shamith L.; Kalaichelvi, V.; Karthikeyan, R.

2018-04-01

TIG Welding is a high quality form of welding which is very popular in industries. It is one of the few types of welding that can be used to join dissimilar metals. Here a weld joint is formed between stainless steel and monel alloy. It is desired to have control over the weld geometry of such a joint through the adjustment of experimental parameters which are welding current, wire feed speed, arc length and the shielding gas flow rate. To facilitate the automation of the same, a model of the welding system is needed. However the underlying welding process is complex and non-linear, and analytical methods are impractical for industrial use. Therefore artificial neural networks (ANN) are explored for developing the model, as they are well-suited for modelling non-linear multi-variate data. Feed-forward neural networks with backpropagation training algorithm are used, and the data for training the ANN taken from experimental work. There are four outputs corresponding to the weld geometry. Different training and testing phases were carried out using MATLAB software and ANN approximates the given data with minimum amount of error.

Groundwater-level prediction using multiple linear regression and artificial neural network techniques: a comparative assessment

NASA Astrophysics Data System (ADS)

Sahoo, Sasmita; Jha, Madan K.

2013-12-01

The potential of multiple linear regression (MLR) and artificial neural network (ANN) techniques in predicting transient water levels over a groundwater basin were compared. MLR and ANN modeling was carried out at 17 sites in Japan, considering all significant inputs: rainfall, ambient temperature, river stage, 11 seasonal dummy variables, and influential lags of rainfall, ambient temperature, river stage and groundwater level. Seventeen site-specific ANN models were developed, using multi-layer feed-forward neural networks trained with Levenberg-Marquardt backpropagation algorithms. The performance of the models was evaluated using statistical and graphical indicators. Comparison of the goodness-of-fit statistics of the MLR models with those of the ANN models indicated that there is better agreement between the ANN-predicted groundwater levels and the observed groundwater levels at all the sites, compared to the MLR. This finding was supported by the graphical indicators and the residual analysis. Thus, it is concluded that the ANN technique is superior to the MLR technique in predicting spatio-temporal distribution of groundwater levels in a basin. However, considering the practical advantages of the MLR technique, it is recommended as an alternative and cost-effective groundwater modeling tool.

Supervised neural network classification of pre-sliced cooked pork ham images using quaternionic singular values.

PubMed

Valous, Nektarios A; Mendoza, Fernando; Sun, Da-Wen; Allen, Paul

2010-03-01

The quaternionic singular value decomposition is a technique to decompose a quaternion matrix (representation of a colour image) into quaternion singular vector and singular value component matrices exposing useful properties. The objective of this study was to use a small portion of uncorrelated singular values, as robust features for the classification of sliced pork ham images, using a supervised artificial neural network classifier. Images were acquired from four qualities of sliced cooked pork ham typically consumed in Ireland (90 slices per quality), having similar appearances. Mahalanobis distances and Pearson product moment correlations were used for feature selection. Six highly discriminating features were used as input to train the neural network. An adaptive feedforward multilayer perceptron classifier was employed to obtain a suitable mapping from the input dataset. The overall correct classification performance for the training, validation and test set were 90.3%, 94.4%, and 86.1%, respectively. The results confirm that the classification performance was satisfactory. Extracting the most informative features led to the recognition of a set of different but visually quite similar textural patterns based on quaternionic singular values. Copyright 2009 Elsevier Ltd. All rights reserved.

Application of principal component regression and artificial neural network in FT-NIR soluble solids content determination of intact pear fruit

NASA Astrophysics Data System (ADS)

Ying, Yibin; Liu, Yande; Fu, Xiaping; Lu, Huishan

2005-11-01

The artificial neural networks (ANNs) have been used successfully in applications such as pattern recognition, image processing, automation and control. However, majority of today's applications of ANNs is back-propagate feed-forward ANN (BP-ANN). In this paper, back-propagation artificial neural networks (BP-ANN) were applied for modeling soluble solid content (SSC) of intact pear from their Fourier transform near infrared (FT-NIR) spectra. One hundred and sixty-four pear samples were used to build the calibration models and evaluate the models predictive ability. The results are compared to the classical calibration approaches, i.e. principal component regression (PCR), partial least squares (PLS) and non-linear PLS (NPLS). The effects of the optimal methods of training parameters on the prediction model were also investigated. BP-ANN combine with principle component regression (PCR) resulted always better than the classical PCR, PLS and Weight-PLS methods, from the point of view of the predictive ability. Based on the results, it can be concluded that FT-NIR spectroscopy and BP-ANN models can be properly employed for rapid and nondestructive determination of fruit internal quality.

One-dimensional inversion of geo-electrical resistivity sounding data using artificial neural networks—a case study

NASA Astrophysics Data System (ADS)

Singh, U. K.; Tiwari, R. K.; Singh, S. B.

2005-02-01

This paper deals with the application of artificial neural networks (ANN) technique for the study of a case history using 1-D inversion of vertical electrical resistivity sounding (VES) data from the Puga valley, Kashmir, India. The study area is important for its rich geothermal resources as well as from the tectonic point of view as it is located near the collision boundary of the Indo-Asian crustal plates. In order to understand the resistivity structure and layer thicknesses, we used here three-layer feedforward neural networks to model and predict measured VES data. Three algorithms, e.g. back-propagation (BP), adaptive back-propagation (ABP) and Levenberg-Marquardt algorithm (LMA) were applied to the synthetic as well as real VES field data and efficiency of supervised training network are compared. Analyses suggest that LMA is computationally faster and give results, which are comparatively more accurate and consistent than BP and ABP. The results obtained using the ANN inversions are remarkably correlated with the available borehole litho-logs. The feasibility study suggests that ANN methods offer an excellent complementary tool for the direct detection of layered resistivity structure.

A Novel Extreme Learning Machine Classification Model for e-Nose Application Based on the Multiple Kernel Approach.

PubMed

Jian, Yulin; Huang, Daoyu; Yan, Jia; Lu, Kun; Huang, Ying; Wen, Tailai; Zeng, Tanyue; Zhong, Shijie; Xie, Qilong

2017-06-19

A novel classification model, named the quantum-behaved particle swarm optimization (QPSO)-based weighted multiple kernel extreme learning machine (QWMK-ELM), is proposed in this paper. Experimental validation is carried out with two different electronic nose (e-nose) datasets. Being different from the existing multiple kernel extreme learning machine (MK-ELM) algorithms, the combination coefficients of base kernels are regarded as external parameters of single-hidden layer feedforward neural networks (SLFNs). The combination coefficients of base kernels, the model parameters of each base kernel, and the regularization parameter are optimized by QPSO simultaneously before implementing the kernel extreme learning machine (KELM) with the composite kernel function. Four types of common single kernel functions (Gaussian kernel, polynomial kernel, sigmoid kernel, and wavelet kernel) are utilized to constitute different composite kernel functions. Moreover, the method is also compared with other existing classification methods: extreme learning machine (ELM), kernel extreme learning machine (KELM), k-nearest neighbors (KNN), support vector machine (SVM), multi-layer perceptron (MLP), radical basis function neural network (RBFNN), and probabilistic neural network (PNN). The results have demonstrated that the proposed QWMK-ELM outperforms the aforementioned methods, not only in precision, but also in efficiency for gas classification.

Sand/cement ratio evaluation on mortar using neural networks and ultrasonic transmission inspection.

PubMed

Molero, M; Segura, I; Izquierdo, M A G; Fuente, J V; Anaya, J J

2009-02-01

The quality and degradation state of building materials can be determined by nondestructive testing (NDT). These materials are composed of a cementitious matrix and particles or fragments of aggregates. Sand/cement ratio (s/c) provides the final material quality; however, the sand content can mask the matrix properties in a nondestructive measurement. Therefore, s/c ratio estimation is needed in nondestructive characterization of cementitious materials. In this study, a methodology to classify the sand content in mortar is presented. The methodology is based on ultrasonic transmission inspection, data reduction, and features extraction by principal components analysis (PCA), and neural network classification. This evaluation is carried out with several mortar samples, which were made while taking into account different cement types and s/c ratios. The estimated s/c ratio is determined by ultrasonic spectral attenuation with three different broadband transducers (0.5, 1, and 2 MHz). Statistical PCA to reduce the dimension of the captured traces has been applied. Feed-forward neural networks (NNs) are trained using principal components (PCs) and their outputs are used to display the estimated s/c ratios in false color images, showing the s/c ratio distribution of the mortar samples.

Excitatory and inhibitory STDP jointly tune feedforward neural circuits to selectively propagate correlated spiking activity

PubMed Central

Kleberg, Florence I.; Fukai, Tomoki; Gilson, Matthieu

2014-01-01

Spike-timing-dependent plasticity (STDP) has been well established between excitatory neurons and several computational functions have been proposed in various neural systems. Despite some recent efforts, however, there is a significant lack of functional understanding of inhibitory STDP (iSTDP) and its interplay with excitatory STDP (eSTDP). Here, we demonstrate by analytical and numerical methods that iSTDP contributes crucially to the balance of excitatory and inhibitory weights for the selection of a specific signaling pathway among other pathways in a feedforward circuit. This pathway selection is based on the high sensitivity of STDP to correlations in spike times, which complements a recent proposal for the role of iSTDP in firing-rate based selection. Our model predicts that asymmetric anti-Hebbian iSTDP exceeds asymmetric Hebbian iSTDP for supporting pathway-specific balance, which we show is useful for propagating transient neuronal responses. Furthermore, we demonstrate how STDPs at excitatory–excitatory, excitatory–inhibitory, and inhibitory–excitatory synapses cooperate to improve the pathway selection. We propose that iSTDP is crucial for shaping the network structure that achieves efficient processing of synchronous spikes. PMID:24847242

A decentralized training algorithm for Echo State Networks in distributed big data applications.

PubMed

Scardapane, Simone; Wang, Dianhui; Panella, Massimo

2016-06-01

The current big data deluge requires innovative solutions for performing efficient inference on large, heterogeneous amounts of information. Apart from the known challenges deriving from high volume and velocity, real-world big data applications may impose additional technological constraints, including the need for a fully decentralized training architecture. While several alternatives exist for training feed-forward neural networks in such a distributed setting, less attention has been devoted to the case of decentralized training of recurrent neural networks (RNNs). In this paper, we propose such an algorithm for a class of RNNs known as Echo State Networks. The algorithm is based on the well-known Alternating Direction Method of Multipliers optimization procedure. It is formulated only in terms of local exchanges between neighboring agents, without reliance on a coordinating node. Additionally, it does not require the communication of training patterns, which is a crucial component in realistic big data implementations. Experimental results on large scale artificial datasets show that it compares favorably with a fully centralized implementation, in terms of speed, efficiency and generalization accuracy. Copyright © 2015 Elsevier Ltd. All rights reserved.

Neural Correlation Is Stimulus Modulated by Feedforward Inhibitory Circuitry

PubMed Central

Middleton, Jason W.; Omar, Cyrus; Doiron, Brent; Simons, Daniel J.

2012-01-01

Correlated variability of neural spiking activity has important consequences for signal processing. How incoming sensory signals shape correlations of population responses remains unclear. Cross-correlations between spiking of different neurons may be particularly consequential in sparsely firing neural populations such as those found in layer 2/3 of sensory cortex. In rat whisker barrel cortex, we found that pairs of excitatory layer 2/3 neurons exhibit similarly low levels of spike count correlation during both spontaneous and sensory-evoked states. The spontaneous activity of excitatory–inhibitory neuron pairs is positively correlated, while sensory stimuli actively decorrelate joint responses. Computational modeling shows how threshold nonlinearities and local inhibition form the basis of a general decorrelating mechanism. We show that inhibitory population activity maintains low correlations in excitatory populations, especially during periods of sensory-evoked coactivation. The role of feedforward inhibition has been previously described in the context of trial-averaged phenomena. Our findings reveal a novel role for inhibition to shape correlations of neural variability and thereby prevent excessive correlations in the face of feedforward sensory-evoked activation. PMID:22238086

Neural network computer simulation of medical aerosols.

PubMed

Richardson, C J; Barlow, D J

1996-06-01

Preliminary investigations have been conducted to assess the potential for using artificial neural networks to simulate aerosol behaviour, with a view to employing this type of methodology in the evaluation and design of pulmonary drug-delivery systems. Details are presented of the general purpose software developed for these tasks; it implements a feed-forward back-propagation algorithm with weight decay and connection pruning, the user having complete run-time control of the network architecture and mode of training. A series of exploratory investigations is then reported in which different network structures and training strategies are assessed in terms of their ability to simulate known patterns of fluid flow in simple model systems. The first of these involves simulations of cellular automata-generated data for fluid flow through a partially obstructed two-dimensional pipe. The artificial neural networks are shown to be highly successful in simulating the behaviour of this simple linear system, but with important provisos relating to the information content of the training data and the criteria used to judge when the network is properly trained. A second set of investigations is then reported in which similar networks are used to simulate patterns of fluid flow through aerosol generation devices, using training data furnished through rigorous computational fluid dynamics modelling. These more complex three-dimensional systems are modelled with equal success. It is concluded that carefully tailored, well trained networks could provide valuable tools not just for predicting but also for analysing the spatial dynamics of pharmaceutical aerosols.

Applications of Artificial Neural Networks in Structural Engineering with Emphasis on Continuum Models

NASA Technical Reports Server (NTRS)

Kapania, Rakesh K.; Liu, Youhua

1998-01-01

The use of continuum models for the analysis of discrete built-up complex aerospace structures is an attractive idea especially at the conceptual and preliminary design stages. But the diversity of available continuum models and hard-to-use qualities of these models have prevented them from finding wide applications. In this regard, Artificial Neural Networks (ANN or NN) may have a great potential as these networks are universal approximators that can realize any continuous mapping, and can provide general mechanisms for building models from data whose input-output relationship can be highly nonlinear. The ultimate aim of the present work is to be able to build high fidelity continuum models for complex aerospace structures using the ANN. As a first step, the concepts and features of ANN are familiarized through the MATLAB NN Toolbox by simulating some representative mapping examples, including some problems in structural engineering. Then some further aspects and lessons learned about the NN training are discussed, including the performances of Feed-Forward and Radial Basis Function NN when dealing with noise-polluted data and the technique of cross-validation. Finally, as an example of using NN in continuum models, a lattice structure with repeating cells is represented by a continuum beam whose properties are provided by neural networks.

Quality assessment and artificial neural networks modeling for characterization of chemical and physical parameters of potable water.

PubMed

Salari, Marjan; Salami Shahid, Esmaeel; Afzali, Seied Hosein; Ehteshami, Majid; Conti, Gea Oliveri; Derakhshan, Zahra; Sheibani, Solmaz Nikbakht

2018-04-22

Today, due to the increase in the population, the growth of industry and the variety of chemical compounds, the quality of drinking water has decreased. Five important river water quality properties such as: dissolved oxygen (DO), total dissolved solids (TDS), total hardness (TH), alkalinity (ALK) and turbidity (TU) were estimated by parameters such as: electric conductivity (EC), temperature (T), and pH that could be measured easily with almost no costs. Simulate water quality parameters were examined with two methods of modeling include mathematical and Artificial Neural Networks (ANN). Mathematical methods are based on polynomial fitting with least square method and ANN modeling algorithms are feed-forward networks. All conditions/circumstances covered by neural network modeling were tested for all parameters in this study, except for Alkalinity. All optimum ANN models developed to simulate water quality parameters had precision value as R-value close to 0.99. The ANN model extended to simulate alkalinity with R-value equals to 0.82. Moreover, Surface fitting techniques were used to refine data sets. Presented models and equations are reliable/useable tools for studying water quality parameters at similar rivers, as a proper replacement for traditional water quality measuring equipment's. Copyright © 2018 Elsevier Ltd. All rights reserved.

Feedback Enhances Feedforward Figure-Ground Segmentation by Changing Firing Mode

PubMed Central

Supèr, Hans; Romeo, August

2011-01-01

In the visual cortex, feedback projections are conjectured to be crucial in figure-ground segregation. However, the precise function of feedback herein is unclear. Here we tested a hypothetical model of reentrant feedback. We used a previous developed 2-layered feedforwardspiking network that is able to segregate figure from ground and included feedback connections. Our computer model data show that without feedback, neurons respond with regular low-frequency (∼9 Hz) bursting to a figure-ground stimulus. After including feedback the firing pattern changed into a regular (tonic) spiking pattern. In this state, we found an extra enhancement of figure responses and a further suppression of background responses resulting in a stronger figure-ground signal. Such push-pull effect was confirmed by comparing the figure-ground responses withthe responses to a homogenous texture. We propose that feedback controlsfigure-ground segregation by influencing the neural firing patterns of feedforward projecting neurons. PMID:21738747

Identification of Correlated GRACE Monthly Harmonic Coefficients Using Pattern Recognition and Neural Networks

NASA Astrophysics Data System (ADS)

Piretzidis, D.; Sra, G.; Sideris, M. G.

2016-12-01

This study explores new methods for identifying correlation errors in harmonic coefficients derived from monthly solutions of the Gravity Recovery and Climate Experiment (GRACE) satellite mission using pattern recognition and neural network algorithms. These correlation errors are evidenced in the differences between monthly solutions and can be suppressed using a de-correlation filter. In all studies so far, the implementation of the de-correlation filter starts from a specific minimum order (i.e., 11 for RL04 and 38 for RL05) until the maximum order of the monthly solution examined. This implementation method has two disadvantages, namely, the omission of filtering correlated coefficients of order less than the minimum order and the filtering of uncorrelated coefficients of order higher than the minimum order. In the first case, the filtered solution is not completely free of correlated errors, whereas the second case results in a monthly solution that suffers from loss of geophysical signal. In the present study, a new method of implementing the de-correlation filter is suggested, by identifying and filtering only the coefficients that show indications of high correlation. Several numerical and geometric properties of the harmonic coefficient series of all orders are examined. Extreme cases of both correlated and uncorrelated coefficients are selected, and their corresponding properties are used to train a two-layer feed-forward neural network. The objective of the neural network is to identify and quantify the correlation by providing the probability of an order of coefficients to be correlated. Results show good performance of the neural network, both in the validation stage of the training procedure and in the subsequent use of the trained network to classify independent coefficients. The neural network is also capable of identifying correlated coefficients even when a small number of training samples and neurons are used (e.g.,100 and 10, respectively).

Computational Account of Spontaneous Activity as a Signature of Predictive Coding

PubMed Central

Koren, Veronika

2017-01-01

Spontaneous activity is commonly observed in a variety of cortical states. Experimental evidence suggested that neural assemblies undergo slow oscillations with Up ad Down states even when the network is isolated from the rest of the brain. Here we show that these spontaneous events can be generated by the recurrent connections within the network and understood as signatures of neural circuits that are correcting their internal representation. A noiseless spiking neural network can represent its input signals most accurately when excitatory and inhibitory currents are as strong and as tightly balanced as possible. However, in the presence of realistic neural noise and synaptic delays, this may result in prohibitively large spike counts. An optimal working regime can be found by considering terms that control firing rates in the objective function from which the network is derived and then minimizing simultaneously the coding error and the cost of neural activity. In biological terms, this is equivalent to tuning neural thresholds and after-spike hyperpolarization. In suboptimal working regimes, we observe spontaneous activity even in the absence of feed-forward inputs. In an all-to-all randomly connected network, the entire population is involved in Up states. In spatially organized networks with local connectivity, Up states spread through local connections between neurons of similar selectivity and take the form of a traveling wave. Up states are observed for a wide range of parameters and have similar statistical properties in both active and quiescent state. In the optimal working regime, Up states are vanishing, leaving place to asynchronous activity, suggesting that this working regime is a signature of maximally efficient coding. Although they result in a massive increase in the firing activity, the read-out of spontaneous Up states is in fact orthogonal to the stimulus representation, therefore interfering minimally with the network function. PMID:28114353

Feature Extraction of Event-Related Potentials Using Wavelets: An Application to Human Performance Monitoring

NASA Technical Reports Server (NTRS)

Trejo, Leonard J.; Shensa, Mark J.; Remington, Roger W. (Technical Monitor)

1998-01-01

This report describes the development and evaluation of mathematical models for predicting human performance from discrete wavelet transforms (DWT) of event-related potentials (ERP) elicited by task-relevant stimuli. The DWT was compared to principal components analysis (PCA) for representation of ERPs in linear regression and neural network models developed to predict a composite measure of human signal detection performance. Linear regression models based on coefficients of the decimated DWT predicted signal detection performance with half as many f ree parameters as comparable models based on PCA scores. In addition, the DWT-based models were more resistant to model degradation due to over-fitting than PCA-based models. Feed-forward neural networks were trained using the backpropagation,-, algorithm to predict signal detection performance based on raw ERPs, PCA scores, or high-power coefficients of the DWT. Neural networks based on high-power DWT coefficients trained with fewer iterations, generalized to new data better, and were more resistant to overfitting than networks based on raw ERPs. Networks based on PCA scores did not generalize to new data as well as either the DWT network or the raw ERP network. The results show that wavelet expansions represent the ERP efficiently and extract behaviorally important features for use in linear regression or neural network models of human performance. The efficiency of the DWT is discussed in terms of its decorrelation and energy compaction properties. In addition, the DWT models provided evidence that a pattern of low-frequency activity (1 to 3.5 Hz) occurring at specific times and scalp locations is a reliable correlate of human signal detection performance.

Feature extraction of event-related potentials using wavelets: an application to human performance monitoring

NASA Technical Reports Server (NTRS)

Trejo, L. J.; Shensa, M. J.

1999-01-01

This report describes the development and evaluation of mathematical models for predicting human performance from discrete wavelet transforms (DWT) of event-related potentials (ERP) elicited by task-relevant stimuli. The DWT was compared to principal components analysis (PCA) for representation of ERPs in linear regression and neural network models developed to predict a composite measure of human signal detection performance. Linear regression models based on coefficients of the decimated DWT predicted signal detection performance with half as many free parameters as comparable models based on PCA scores. In addition, the DWT-based models were more resistant to model degradation due to over-fitting than PCA-based models. Feed-forward neural networks were trained using the backpropagation algorithm to predict signal detection performance based on raw ERPs, PCA scores, or high-power coefficients of the DWT. Neural networks based on high-power DWT coefficients trained with fewer iterations, generalized to new data better, and were more resistant to overfitting than networks based on raw ERPs. Networks based on PCA scores did not generalize to new data as well as either the DWT network or the raw ERP network. The results show that wavelet expansions represent the ERP efficiently and extract behaviorally important features for use in linear regression or neural network models of human performance. The efficiency of the DWT is discussed in terms of its decorrelation and energy compaction properties. In addition, the DWT models provided evidence that a pattern of low-frequency activity (1 to 3.5 Hz) occurring at specific times and scalp locations is a reliable correlate of human signal detection performance. Copyright 1999 Academic Press.

Optimal feature selection using a modified differential evolution algorithm and its effectiveness for prediction of heart disease.

PubMed

Vivekanandan, T; Sriman Narayana Iyengar, N Ch

2017-11-01

Enormous data growth in multiple domains has posed a great challenge for data processing and analysis techniques. In particular, the traditional record maintenance strategy has been replaced in the healthcare system. It is vital to develop a model that is able to handle the huge amount of e-healthcare data efficiently. In this paper, the challenging tasks of selecting critical features from the enormous set of available features and diagnosing heart disease are carried out. Feature selection is one of the most widely used pre-processing steps in classification problems. A modified differential evolution (DE) algorithm is used to perform feature selection for cardiovascular disease and optimization of selected features. Of the 10 available strategies for the traditional DE algorithm, the seventh strategy, which is represented by DE/rand/2/exp, is considered for comparative study. The performance analysis of the developed modified DE strategy is given in this paper. With the selected critical features, prediction of heart disease is carried out using fuzzy AHP and a feed-forward neural network. Various performance measures of integrating the modified differential evolution algorithm with fuzzy AHP and a feed-forward neural network in the prediction of heart disease are evaluated in this paper. The accuracy of the proposed hybrid model is 83%, which is higher than that of some other existing models. In addition, the prediction time of the proposed hybrid model is also evaluated and has shown promising results. Copyright © 2017 Elsevier Ltd. All rights reserved.

Central Control of Brown Adipose Tissue Thermogenesis

PubMed Central

Morrison, Shaun F.; Madden, Christopher J.; Tupone, Domenico

2011-01-01

Thermogenesis, the production of heat energy, is an essential component of the homeostatic repertoire to maintain body temperature during the challenge of low environmental temperature and plays a key role in elevating body temperature during the febrile response to infection. Mitochondrial oxidation in brown adipose tissue (BAT) is a significant source of neurally regulated metabolic heat production in many species from mouse to man. BAT thermogenesis is regulated by neural networks in the central nervous system which responds to feedforward afferent signals from cutaneous and core body thermoreceptors and to feedback signals from brain thermosensitive neurons to activate BAT sympathetic nerve activity. This review summarizes the research leading to a model of the feedforward reflex pathway through which environmental cold stimulates BAT thermogenesis and includes the influence on this thermoregulatory network of the pyrogenic mediator, prostaglandin E2, to increase body temperature during fever. The cold thermal afferent circuit from cutaneous thermal receptors, through second-order thermosensory neurons in the dorsal horn of the spinal cord ascends to activate neurons in the lateral parabrachial nucleus which drive GABAergic interneurons in the preoptic area (POA) to inhibit warm-sensitive, inhibitory output neurons of the POA. The resulting disinhibition of BAT thermogenesis-promoting neurons in the dorsomedial hypothalamus activates BAT sympathetic premotor neurons in the rostral ventromedial medulla, including the rostral raphe pallidus, which provide excitatory, and possibly disinhibitory, inputs to spinal sympathetic circuits to drive BAT thermogenesis. Other recently recognized central sites influencing BAT thermogenesis and energy expenditure are also described. PMID:22389645

Role of the cerebellum in reaching movements in humans. II. A neural model of the intermediate cerebellum.

PubMed

Schweighofer, N; Spoelstra, J; Arbib, M A; Kawato, M

1998-01-01

The cerebellum is essential for the control of multijoint movements; when the cerebellum is lesioned, the performance error is more than the summed errors produced by single joints. In the companion paper (Schweighofer et al., 1998), a functional anatomical model for visually guided arm movement was proposed. The model comprised a basic feedforward/feedback controller with realistic transmission delays and was connected to a two-link, six-muscle, planar arm. In the present study, we examined the role of the cerebellum in reaching movements by embedding a novel, detailed cerebellar neural network in this functional control model. We could derive realistic cerebellar inputs and the role of the cerebellum in learning to control the arm was assessed. This cerebellar network learned the part of the inverse dynamics of the arm not provided by the basic feedforward/feedback controller. Despite realistically low inferior olive firing rates and noisy mossy fibre inputs, the model could reduce the error between intended and planned movements. The responses of the different cell groups were comparable to those of biological cell groups. In particular, the modelled Purkinje cells exhibited directional tuning after learning and the parallel fibres, due to their length, provide Purkinje cells with the input required for this coordination task. The inferior olive responses contained two different components; the earlier response, locked to movement onset, was always present and the later response disappeared after learning. These results support the theory that the cerebellum is involved in motor learning.

Recursive least-squares learning algorithms for neural networks

NASA Astrophysics Data System (ADS)

Lewis, Paul S.; Hwang, Jenq N.

1990-11-01

This paper presents the development of a pair of recursive least squares (ItLS) algorithms for online training of multilayer perceptrons which are a class of feedforward artificial neural networks. These algorithms incorporate second order information about the training error surface in order to achieve faster learning rates than are possible using first order gradient descent algorithms such as the generalized delta rule. A least squares formulation is derived from a linearization of the training error function. Individual training pattern errors are linearized about the network parameters that were in effect when the pattern was presented. This permits the recursive solution of the least squares approximation either via conventional RLS recursions or by recursive QR decomposition-based techniques. The computational complexity of the update is 0(N2) where N is the number of network parameters. This is due to the estimation of the N x N inverse Hessian matrix. Less computationally intensive approximations of the ilLS algorithms can be easily derived by using only block diagonal elements of this matrix thereby partitioning the learning into independent sets. A simulation example is presented in which a neural network is trained to approximate a two dimensional Gaussian bump. In this example RLS training required an order of magnitude fewer iterations on average (527) than did training with the generalized delta rule (6 1 BACKGROUND Artificial neural networks (ANNs) offer an interesting and potentially useful paradigm for signal processing and pattern recognition. The majority of ANN applications employ the feed-forward multilayer perceptron (MLP) network architecture in which network parameters are " trained" by a supervised learning algorithm employing the generalized delta rule (GDIt) [1 2]. The GDR algorithm approximates a fixed step steepest descent algorithm using derivatives computed by error backpropagatiori. The GDII algorithm is sometimes referred to as the backpropagation algorithm. However in this paper we will use the term backpropagation to refer only to the process of computing error derivatives. While multilayer perceptrons provide a very powerful nonlinear modeling capability GDR training can be very slow and inefficient. In linear adaptive filtering the analog of the GDR algorithm is the leastmean- squares (LMS) algorithm. Steepest descent-based algorithms such as GDR or LMS are first order because they use only first derivative or gradient information about the training error to be minimized. To speed up the training process second order algorithms may be employed that take advantage of second derivative or Hessian matrix information. Second order information can be incorporated into MLP training in different ways. In many applications especially in the area of pattern recognition the training set is finite. In these cases block learning can be applied using standard nonlinear optimization techniques [3 4 5].

Meta-modeling of the pesticide fate model MACRO for groundwater exposure assessments using artificial neural networks

NASA Astrophysics Data System (ADS)

Stenemo, Fredrik; Lindahl, Anna M. L.; Gärdenäs, Annemieke; Jarvis, Nicholas

2007-08-01

Several simple index methods that use easily accessible data have been developed and included in decision-support systems to estimate pesticide leaching across larger areas. However, these methods often lack important process descriptions (e.g. macropore flow), which brings into question their reliability. Descriptions of macropore flow have been included in simulation models, but these are too complex and demanding for spatial applications. To resolve this dilemma, a neural network simulation meta-model of the dual-permeability macropore flow model MACRO was created for pesticide groundwater exposure assessment. The model was parameterized using pedotransfer functions that require as input the clay and sand content of the topsoil and subsoil, and the topsoil organic carbon content. The meta-model also requires the topsoil pesticide half-life and the soil organic carbon sorption coefficient as input. A fully connected feed-forward multilayer perceptron classification network with two hidden layers, linked to fully connected feed-forward multilayer perceptron neural networks with one hidden layer, trained on sub-sets of the target variable, was shown to be a suitable meta-model for the intended purpose. A Fourier amplitude sensitivity test showed that the model output (the 80th percentile average yearly pesticide concentration at 1 m depth for a 20 year simulation period) was sensitive to all input parameters. The two input parameters related to pesticide characteristics (i.e. soil organic carbon sorption coefficient and topsoil pesticide half-life) were the most influential, but texture in the topsoil was also quite important since it was assumed to control the mass exchange coefficient that regulates the strength of macropore flow. This is in contrast to models based on the advection-dispersion equation where soil texture is relatively unimportant. The use of the meta-model is exemplified with a case-study where the spatial variability of pesticide leaching is mapped for a small field. It was shown that the area of the field that contributes most to leaching depends on the properties of the compound in question. It is concluded that the simulation meta-model of MACRO should prove useful for mapping relative pesticide leaching risks at large scales.

New model for prediction binary mixture of antihistamine decongestant using artificial neural networks and least squares support vector machine by spectrophotometry method

NASA Astrophysics Data System (ADS)

Mofavvaz, Shirin; Sohrabi, Mahmoud Reza; Nezamzadeh-Ejhieh, Alireza

2017-07-01

In the present study, artificial neural networks (ANNs) and least squares support vector machines (LS-SVM) as intelligent methods based on absorption spectra in the range of 230-300 nm have been used for determination of antihistamine decongestant contents. In the first step, one type of network (feed-forward back-propagation) from the artificial neural network with two different training algorithms, Levenberg-Marquardt (LM) and gradient descent with momentum and adaptive learning rate back-propagation (GDX) algorithm, were employed and their performance was evaluated. The performance of the LM algorithm was better than the GDX algorithm. In the second one, the radial basis network was utilized and results compared with the previous network. In the last one, the other intelligent method named least squares support vector machine was proposed to construct the antihistamine decongestant prediction model and the results were compared with two of the aforementioned networks. The values of the statistical parameters mean square error (MSE), Regression coefficient (R2), correlation coefficient (r) and also mean recovery (%), relative standard deviation (RSD) used for selecting the best model between these methods. Moreover, the proposed methods were compared to the high- performance liquid chromatography (HPLC) as a reference method. One way analysis of variance (ANOVA) test at the 95% confidence level applied to the comparison results of suggested and reference methods that there were no significant differences between them.

Short-term forecasting of electric loads using nonlinear autoregressive artificial neural networks with exogenous vector inputs

DOE PAGES

Buitrago, Jaime; Asfour, Shihab

2017-01-01

Short-term load forecasting is crucial for the operations planning of an electrical grid. Forecasting the next 24 h of electrical load in a grid allows operators to plan and optimize their resources. The purpose of this study is to develop a more accurate short-term load forecasting method utilizing non-linear autoregressive artificial neural networks (ANN) with exogenous multi-variable input (NARX). The proposed implementation of the network is new: the neural network is trained in open-loop using actual load and weather data, and then, the network is placed in closed-loop to generate a forecast using the predicted load as the feedback input.more » Unlike the existing short-term load forecasting methods using ANNs, the proposed method uses its own output as the input in order to improve the accuracy, thus effectively implementing a feedback loop for the load, making it less dependent on external data. Using the proposed framework, mean absolute percent errors in the forecast in the order of 1% have been achieved, which is a 30% improvement on the average error using feedforward ANNs, ARMAX and state space methods, which can result in large savings by avoiding commissioning of unnecessary power plants. Finally, the New England electrical load data are used to train and validate the forecast prediction.« less

Short-term forecasting of electric loads using nonlinear autoregressive artificial neural networks with exogenous vector inputs

DOE Office of Scientific and Technical Information (OSTI.GOV)

Buitrago, Jaime; Asfour, Shihab

Short-term load forecasting is crucial for the operations planning of an electrical grid. Forecasting the next 24 h of electrical load in a grid allows operators to plan and optimize their resources. The purpose of this study is to develop a more accurate short-term load forecasting method utilizing non-linear autoregressive artificial neural networks (ANN) with exogenous multi-variable input (NARX). The proposed implementation of the network is new: the neural network is trained in open-loop using actual load and weather data, and then, the network is placed in closed-loop to generate a forecast using the predicted load as the feedback input.more » Unlike the existing short-term load forecasting methods using ANNs, the proposed method uses its own output as the input in order to improve the accuracy, thus effectively implementing a feedback loop for the load, making it less dependent on external data. Using the proposed framework, mean absolute percent errors in the forecast in the order of 1% have been achieved, which is a 30% improvement on the average error using feedforward ANNs, ARMAX and state space methods, which can result in large savings by avoiding commissioning of unnecessary power plants. Finally, the New England electrical load data are used to train and validate the forecast prediction.« less

Neural Oscillations and Synchrony in Brain Dysfunction and Neuropsychiatric Disorders: It's About Time.

PubMed

Mathalon, Daniel H; Sohal, Vikaas S

2015-08-01

Neural oscillations are rhythmic fluctuations over time in the activity or excitability of single neurons, local neuronal populations or "assemblies," and/or multiple regionally distributed neuronal assemblies. Synchronized oscillations among large numbers of neurons are evident in electrocorticographic, electroencephalographic, magnetoencephalographic, and local field potential recordings and are generally understood to depend on inhibition that paces assemblies of excitatory neurons to produce alternating temporal windows of reduced and increased excitability. Synchronization of neural oscillations is supported by the extensive networks of local and long-range feedforward and feedback bidirectional connections between neurons. Here, we review some of the major methods and measures used to characterize neural oscillations, with a focus on gamma oscillations. Distinctions are drawn between stimulus-independent oscillations recorded during resting states or intervals between task events, stimulus-induced oscillations that are time locked but not phase locked to stimuli, and stimulus-evoked oscillations that are both time and phase locked to stimuli. Synchrony of oscillations between recording sites, and between the amplitudes and phases of oscillations of different frequencies (cross-frequency coupling), is described and illustrated. Molecular mechanisms underlying gamma oscillations are also reviewed. Ultimately, understanding the temporal organization of neuronal network activity, including interactions between neural oscillations, is critical for elucidating brain dysfunction in neuropsychiatric disorders.

Recurrent Network models of sequence generation and memory

PubMed Central

Rajan, Kanaka; Harvey, Christopher D; Tank, David W

2016-01-01

SUMMARY Sequential activation of neurons is a common feature of network activity during a variety of behaviors, including working memory and decision making. Previous network models for sequences and memory emphasized specialized architectures in which a principled mechanism is pre-wired into their connectivity. Here, we demonstrate that starting from random connectivity and modifying a small fraction of connections, a largely disordered recurrent network can produce sequences and implement working memory efficiently. We use this process, called Partial In-Network training (PINning), to model and match cellular-resolution imaging data from the posterior parietal cortex during a virtual memory-guided two-alternative forced choice task [Harvey, Coen and Tank, 2012]. Analysis of the connectivity reveals that sequences propagate by the cooperation between recurrent synaptic interactions and external inputs, rather than through feedforward or asymmetric connections. Together our results suggest that neural sequences may emerge through learning from largely unstructured network architectures. PMID:26971945

Acquisition and Neural Network Prediction of 3D Deformable Object Shape Using a Kinect and a Force-Torque Sensor.

PubMed

Tawbe, Bilal; Cretu, Ana-Maria

2017-05-11

The realistic representation of deformations is still an active area of research, especially for deformable objects whose behavior cannot be simply described in terms of elasticity parameters. This paper proposes a data-driven neural-network-based approach for capturing implicitly and predicting the deformations of an object subject to external forces. Visual data, in the form of 3D point clouds gathered by a Kinect sensor, is collected over an object while forces are exerted by means of the probing tip of a force-torque sensor. A novel approach based on neural gas fitting is proposed to describe the particularities of a deformation over the selectively simplified 3D surface of the object, without requiring knowledge of the object material. An alignment procedure, a distance-based clustering, and inspiration from stratified sampling support this process. The resulting representation is denser in the region of the deformation (an average of 96.6% perceptual similarity with the collected data in the deformed area), while still preserving the object's overall shape (86% similarity over the entire surface) and only using on average of 40% of the number of vertices in the mesh. A series of feedforward neural networks is then trained to predict the mapping between the force parameters characterizing the interaction with the object and the change in the object shape, as captured by the fitted neural gas nodes. This series of networks allows for the prediction of the deformation of an object when subject to unknown interactions.

Biological engineering applications of feedforward neural networks designed and parameterized by genetic algorithms.

PubMed

Ferentinos, Konstantinos P

2005-09-01

Two neural network (NN) applications in the field of biological engineering are developed, designed and parameterized by an evolutionary method based on the evolutionary process of genetic algorithms. The developed systems are a fault detection NN model and a predictive modeling NN system. An indirect or 'weak specification' representation was used for the encoding of NN topologies and training parameters into genes of the genetic algorithm (GA). Some a priori knowledge of the demands in network topology for specific application cases is required by this approach, so that the infinite search space of the problem is limited to some reasonable degree. Both one-hidden-layer and two-hidden-layer network architectures were explored by the GA. Except for the network architecture, each gene of the GA also encoded the type of activation functions in both hidden and output nodes of the NN and the type of minimization algorithm that was used by the backpropagation algorithm for the training of the NN. Both models achieved satisfactory performance, while the GA system proved to be a powerful tool that can successfully replace the problematic trial-and-error approach that is usually used for these tasks.

Comparison of Multiple Linear Regressions and Neural Networks based QSAR models for the design of new antitubercular compounds.

PubMed

Ventura, Cristina; Latino, Diogo A R S; Martins, Filomena

2013-01-01

The performance of two QSAR methodologies, namely Multiple Linear Regressions (MLR) and Neural Networks (NN), towards the modeling and prediction of antitubercular activity was evaluated and compared. A data set of 173 potentially active compounds belonging to the hydrazide family and represented by 96 descriptors was analyzed. Models were built with Multiple Linear Regressions (MLR), single Feed-Forward Neural Networks (FFNNs), ensembles of FFNNs and Associative Neural Networks (AsNNs) using four different data sets and different types of descriptors. The predictive ability of the different techniques used were assessed and discussed on the basis of different validation criteria and results show in general a better performance of AsNNs in terms of learning ability and prediction of antitubercular behaviors when compared with all other methods. MLR have, however, the advantage of pinpointing the most relevant molecular characteristics responsible for the behavior of these compounds against Mycobacterium tuberculosis. The best results for the larger data set (94 compounds in training set and 18 in test set) were obtained with AsNNs using seven descriptors (R(2) of 0.874 and RMSE of 0.437 against R(2) of 0.845 and RMSE of 0.472 in MLRs, for test set). Counter-Propagation Neural Networks (CPNNs) were trained with the same data sets and descriptors. From the scrutiny of the weight levels in each CPNN and the information retrieved from MLRs, a rational design of potentially active compounds was attempted. Two new compounds were synthesized and tested against M. tuberculosis showing an activity close to that predicted by the majority of the models. Copyright © 2013 Elsevier Masson SAS. All rights reserved.

Feedforward Inhibition and Synaptic Scaling – Two Sides of the Same Coin?

PubMed Central

Lücke, Jörg

2012-01-01

Feedforward inhibition and synaptic scaling are important adaptive processes that control the total input a neuron can receive from its afferents. While often studied in isolation, the two have been reported to co-occur in various brain regions. The functional implications of their interactions remain unclear, however. Based on a probabilistic modeling approach, we show here that fast feedforward inhibition and synaptic scaling interact synergistically during unsupervised learning. In technical terms, we model the input to a neural circuit using a normalized mixture model with Poisson noise. We demonstrate analytically and numerically that, in the presence of lateral inhibition introducing competition between different neurons, Hebbian plasticity and synaptic scaling approximate the optimal maximum likelihood solutions for this model. Our results suggest that, beyond its conventional use as a mechanism to remove undesired pattern variations, input normalization can make typical neural interaction and learning rules optimal on the stimulus subspace defined through feedforward inhibition. Furthermore, learning within this subspace is more efficient in practice, as it helps avoid locally optimal solutions. Our results suggest a close connection between feedforward inhibition and synaptic scaling which may have important functional implications for general cortical processing. PMID:22457610

Feedforward inhibition and synaptic scaling--two sides of the same coin?

PubMed

Keck, Christian; Savin, Cristina; Lücke, Jörg

2012-01-01

Feedforward inhibition and synaptic scaling are important adaptive processes that control the total input a neuron can receive from its afferents. While often studied in isolation, the two have been reported to co-occur in various brain regions. The functional implications of their interactions remain unclear, however. Based on a probabilistic modeling approach, we show here that fast feedforward inhibition and synaptic scaling interact synergistically during unsupervised learning. In technical terms, we model the input to a neural circuit using a normalized mixture model with Poisson noise. We demonstrate analytically and numerically that, in the presence of lateral inhibition introducing competition between different neurons, Hebbian plasticity and synaptic scaling approximate the optimal maximum likelihood solutions for this model. Our results suggest that, beyond its conventional use as a mechanism to remove undesired pattern variations, input normalization can make typical neural interaction and learning rules optimal on the stimulus subspace defined through feedforward inhibition. Furthermore, learning within this subspace is more efficient in practice, as it helps avoid locally optimal solutions. Our results suggest a close connection between feedforward inhibition and synaptic scaling which may have important functional implications for general cortical processing.

The structure-AChE inhibitory activity relationships study in a series of pyridazine analogues.

PubMed

Saracoglu, M; Kandemirli, F

2009-07-01

The structure-activity relationships (SAR) are investigated by means of the Electronic-Topological Method (ETM) followed by the Neural Networks application (ETM-NN) for a class of anti-cholinesterase inhibitors (AChE, 53 molecules) being pyridazine derivatives. AChE activities of the series were measured in IC(50) units, and relative to the activity levels, the series was partitioned into classes of active and inactive compounds. Based on pharmacophores and antipharmacophores calculated by the ETM-software as sub-matrices containing important spatial and electronic characteristics, a system for the activity prognostication is developed. Input data for the ETM were taken as the results of conformational and quantum-mechanics calculations. To predict the activity, we used one of the most well known neural networks, namely, the feed-forward neural networks (FFNNs) trained with the back propagation algorithm. The supervised learning was performed using a variant of FFNN known as the Associative Neural Networks (ASNN). The result of the testing revealed that the high ETM's ability of predicting both activity and inactivity of potential AChE inhibitors. Analysis of HOMOs for the compounds containing Ph1 and APh1 has shown that atoms with the highest values of the atomic orbital coefficients are mainly those atoms that enter into the pharmacophores. Thus, the set of pharmacophores and antipharmacophores found as the result of this study forms a basis for a system of the anti-cholinesterase activity prediction.

A neuro-fuzzy architecture for real-time applications

NASA Technical Reports Server (NTRS)

Ramamoorthy, P. A.; Huang, Song

1992-01-01

Neural networks and fuzzy expert systems perform the same task of functional mapping using entirely different approaches. Each approach has certain unique features. The ability to learn specific input-output mappings from large input/output data possibly corrupted by noise and the ability to adapt or continue learning are some important features of neural networks. Fuzzy expert systems are known for their ability to deal with fuzzy information and incomplete/imprecise data in a structured, logical way. Since both of these techniques implement the same task (that of functional mapping--we regard 'inferencing' as one specific category under this class), a fusion of the two concepts that retains their unique features while overcoming their individual drawbacks will have excellent applications in the real world. In this paper, we arrive at a new architecture by fusing the two concepts. The architecture has the trainability/adaptibility (based on input/output observations) property of the neural networks and the architectural features that are unique to fuzzy expert systems. It also does not require specific information such as fuzzy rules, defuzzification procedure used, etc., though any such information can be integrated into the architecture. We show that this architecture can provide better performance than is possible from a single two or three layer feedforward neural network. Further, we show that this new architecture can be used as an efficient vehicle for hardware implementation of complex fuzzy expert systems for real-time applications. A numerical example is provided to show the potential of this approach.

Video Super-Resolution via Bidirectional Recurrent Convolutional Networks.

PubMed

Huang, Yan; Wang, Wei; Wang, Liang

2018-04-01

Super resolving a low-resolution video, namely video super-resolution (SR), is usually handled by either single-image SR or multi-frame SR. Single-Image SR deals with each video frame independently, and ignores intrinsic temporal dependency of video frames which actually plays a very important role in video SR. Multi-Frame SR generally extracts motion information, e.g., optical flow, to model the temporal dependency, but often shows high computational cost. Considering that recurrent neural networks (RNNs) can model long-term temporal dependency of video sequences well, we propose a fully convolutional RNN named bidirectional recurrent convolutional network for efficient multi-frame SR. Different from vanilla RNNs, 1) the commonly-used full feedforward and recurrent connections are replaced with weight-sharing convolutional connections. So they can greatly reduce the large number of network parameters and well model the temporal dependency in a finer level, i.e., patch-based rather than frame-based, and 2) connections from input layers at previous timesteps to the current hidden layer are added by 3D feedforward convolutions, which aim to capture discriminate spatio-temporal patterns for short-term fast-varying motions in local adjacent frames. Due to the cheap convolutional operations, our model has a low computational complexity and runs orders of magnitude faster than other multi-frame SR methods. With the powerful temporal dependency modeling, our model can super resolve videos with complex motions and achieve well performance.

Deep learning beyond Lefschetz thimbles

NASA Astrophysics Data System (ADS)

Alexandru, Andrei; Bedaque, Paulo F.; Lamm, Henry; Lawrence, Scott

2017-11-01

The generalized thimble method to treat field theories with sign problems requires repeatedly solving the computationally expensive holomorphic flow equations. We present a machine learning technique to bypass this problem. The central idea is to obtain a few field configurations via the flow equations to train a feed-forward neural network. The trained network defines a new manifold of integration which reduces the sign problem and can be rapidly sampled. We present results for the 1 +1 dimensional Thirring model with Wilson fermions on sizable lattices. In addition to the gain in speed, the parametrization of the integration manifold we use avoids the "trapping" of Monte Carlo chains which plagues large-flow calculations, a considerable shortcoming of the previous attempts.

Weight-elimination neural networks applied to coronary surgery mortality prediction.

PubMed

Ennett, Colleen M; Frize, Monique

2003-06-01

The objective was to assess the effectiveness of the weight-elimination cost function in improving classification performance of artificial neural networks (ANNs) and to observe how changing the a priori distribution of the training set affects network performance. Backpropagation feedforward ANNs with and without weight-elimination estimated mortality for coronary artery surgery patients. The ANNs were trained and tested on cases with 32 input variables describing the patient's medical history; the output variable was in-hospital mortality (mortality rates: training 3.7%, test 3.8%). Artificial training sets with mortality rates of 20%, 50%, and 80% were created to observe the impact of training with a higher-than-normal prevalence. When the results were averaged, weight-elimination networks achieved higher sensitivity rates than those without weight-elimination. Networks trained on higher-than-normal prevalence achieved higher sensitivity rates at the cost of lower specificity and correct classification. The weight-elimination cost function can improve the classification performance when the network is trained with a higher-than-normal prevalence. A network trained with a moderately high artificial mortality rate (artificial mortality rate of 20%) can improve the sensitivity of the model without significantly affecting other aspects of the model's performance. The ANN mortality model achieved comparable performance as additive and statistical models for coronary surgery mortality estimation in the literature.

A Fast Neural Network Approach to Predict Lung Tumor Motion during Respiration for Radiation Therapy Applications

PubMed Central

Slama, Matous; Benes, Peter M.; Bila, Jiri

2015-01-01

During radiotherapy treatment for thoracic and abdomen cancers, for example, lung cancers, respiratory motion moves the target tumor and thus badly affects the accuracy of radiation dose delivery into the target. A real-time image-guided technique can be used to monitor such lung tumor motion for accurate dose delivery, but the system latency up to several hundred milliseconds for repositioning the radiation beam also affects the accuracy. In order to compensate the latency, neural network prediction technique with real-time retraining can be used. We have investigated real-time prediction of 3D time series of lung tumor motion on a classical linear model, perceptron model, and on a class of higher-order neural network model that has more attractive attributes regarding its optimization convergence and computational efficiency. The implemented static feed-forward neural architectures are compared when using gradient descent adaptation and primarily the Levenberg-Marquardt batch algorithm as the ones of the most common and most comprehensible learning algorithms. The proposed technique resulted in fast real-time retraining, so the total computational time on a PC platform was equal to or even less than the real treatment time. For one-second prediction horizon, the proposed techniques achieved accuracy less than one millimeter of 3D mean absolute error in one hundred seconds of total treatment time. PMID:25893194

A fast neural network approach to predict lung tumor motion during respiration for radiation therapy applications.

PubMed

Bukovsky, Ivo; Homma, Noriyasu; Ichiji, Kei; Cejnek, Matous; Slama, Matous; Benes, Peter M; Bila, Jiri

2015-01-01

During radiotherapy treatment for thoracic and abdomen cancers, for example, lung cancers, respiratory motion moves the target tumor and thus badly affects the accuracy of radiation dose delivery into the target. A real-time image-guided technique can be used to monitor such lung tumor motion for accurate dose delivery, but the system latency up to several hundred milliseconds for repositioning the radiation beam also affects the accuracy. In order to compensate the latency, neural network prediction technique with real-time retraining can be used. We have investigated real-time prediction of 3D time series of lung tumor motion on a classical linear model, perceptron model, and on a class of higher-order neural network model that has more attractive attributes regarding its optimization convergence and computational efficiency. The implemented static feed-forward neural architectures are compared when using gradient descent adaptation and primarily the Levenberg-Marquardt batch algorithm as the ones of the most common and most comprehensible learning algorithms. The proposed technique resulted in fast real-time retraining, so the total computational time on a PC platform was equal to or even less than the real treatment time. For one-second prediction horizon, the proposed techniques achieved accuracy less than one millimeter of 3D mean absolute error in one hundred seconds of total treatment time.

Estimating surface longwave radiative fluxes from satellites utilizing artificial neural networks

NASA Astrophysics Data System (ADS)

Nussbaumer, Eric A.; Pinker, Rachel T.

2012-04-01

A novel approach for calculating downwelling surface longwave (DSLW) radiation under all sky conditions is presented. The DSLW model (hereafter, DSLW/UMD v2) similarly to its predecessor, DSLW/UMD v1, is driven with a combination of Moderate Resolution Imaging Spectroradiometer (MODIS) level-3 cloud parameters and information from the European Centre for Medium-Range Weather Forecasts (ECMWF) ERA-Interim model. To compute the clear sky component of DSLW a two layer feed-forward artificial neural network with sigmoid hidden neurons and linear output neurons is implemented; it is trained with simulations derived from runs of the Rapid Radiative Transfer Model (RRTM). When computing the cloud contribution to DSLW, the cloud base temperature is estimated by using an independent artificial neural network approach of similar architecture as previously mentioned, and parameterizations. The cloud base temperature neural network is trained using spatially and temporally co-located MODIS and CloudSat Cloud Profiling Radar (CPR) and the Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observation (CALIPSO) Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) observations. Daily average estimates of DSLW from 2003 to 2009 are compared against ground measurements from the Baseline Surface Radiation Network (BSRN) giving an overall correlation coefficient of 0.98, root mean square error (rmse) of 15.84 W m-2, and a bias of -0.39 W m-2. This is an improvement over an earlier version of the model (DSLW/UMD v1) which for the same time period has an overall correlation coefficient 0.97 rmse of 17.27 W m-2, and bias of 0.73 W m-2.

Actividad solar del ciclo 23. Predicción del máximo y fase decreciente utilizando redes neuronales

NASA Astrophysics Data System (ADS)

Parodi, M. A.; Ceccatto, H. A.; Piacentini, R. D.; García, P. J.

Different methods have been proposed in order to predict the maximum amplitude of solar cycles, either as a consequence of the intrinsic importance of this event and because of its relation with solar storms and possible effects upon satellites, communication systems, etc. In this work, a neural network solar activity prediction is presented, measured through the sunspot number (SSN). The 16-units neural network, with a 12:3:1 architecture, was trained in a ``feed-forward" propagation way and learning by the so called ``back propagation rule". The annual mean SSN data in the 1700-1975 and 1987-1998 periods were used as the training set. The solar cycle 21 (1976-1986) was taken as the cross-validation data set. After performing the network training we obtained a prediction of the maximum annual mean for the current solar cycle 23, SSNmax= 135 ±17 at the year 2000, which is 13% smaller than the International Consensus Commitee's mean maximum prediction obtained through ``precursor techniques". On the other hand, our prediction is only about 4% smaller than the Consensus's neural network mean prediction. A ``multiple step" prediction technique was also performed and SSN annual mean predicted values for the near-maximum (from the present year 1999 to beyond the maximum) and the declining activity of solar cycle 23 are presented in this work. The sensibility of predictions is also tested. To do so, we changed the interval width and comparated our results with those of a previous neural network prediction and those of others authors using differents methods.

Predicting healthcare associated infections using patients' experiences

NASA Astrophysics Data System (ADS)

Pratt, Michael A.; Chu, Henry

2016-05-01

Healthcare associated infections (HAI) are a major threat to patient safety and are costly to health systems. Our goal is to predict the HAI performance of a hospital using the patients' experience responses as input. We use four classifiers, viz. random forest, naive Bayes, artificial feedforward neural networks, and the support vector machine, to perform the prediction of six types of HAI. The six types include blood stream, urinary tract, surgical site, and intestinal infections. Experiments show that the random forest and support vector machine perform well across the six types of HAI.

SKYNET: an efficient and robust neural network training tool for machine learning in astronomy

NASA Astrophysics Data System (ADS)

Graff, Philip; Feroz, Farhan; Hobson, Michael P.; Lasenby, Anthony

2014-06-01

We present the first public release of our generic neural network training algorithm, called SKYNET. This efficient and robust machine learning tool is able to train large and deep feed-forward neural networks, including autoencoders, for use in a wide range of supervised and unsupervised learning applications, such as regression, classification, density estimation, clustering and dimensionality reduction. SKYNET uses a `pre-training' method to obtain a set of network parameters that has empirically been shown to be close to a good solution, followed by further optimization using a regularized variant of Newton's method, where the level of regularization is determined and adjusted automatically; the latter uses second-order derivative information to improve convergence, but without the need to evaluate or store the full Hessian matrix, by using a fast approximate method to calculate Hessian-vector products. This combination of methods allows for the training of complicated networks that are difficult to optimize using standard backpropagation techniques. SKYNET employs convergence criteria that naturally prevent overfitting, and also includes a fast algorithm for estimating the accuracy of network outputs. The utility and flexibility of SKYNET are demonstrated by application to a number of toy problems, and to astronomical problems focusing on the recovery of structure from blurred and noisy images, the identification of gamma-ray bursters, and the compression and denoising of galaxy images. The SKYNET software, which is implemented in standard ANSI C and fully parallelized using MPI, is available at http://www.mrao.cam.ac.uk/software/skynet/.

Estimation of effective connectivity using multi-layer perceptron artificial neural network.

PubMed

Talebi, Nasibeh; Nasrabadi, Ali Motie; Mohammad-Rezazadeh, Iman

2018-02-01

Studies on interactions between brain regions estimate effective connectivity, (usually) based on the causality inferences made on the basis of temporal precedence. In this study, the causal relationship is modeled by a multi-layer perceptron feed-forward artificial neural network, because of the ANN's ability to generate appropriate input-output mapping and to learn from training examples without the need of detailed knowledge of the underlying system. At any time instant, the past samples of data are placed in the network input, and the subsequent values are predicted at its output. To estimate the strength of interactions, the measure of " Causality coefficient " is defined based on the network structure, the connecting weights and the parameters of hidden layer activation function. Simulation analysis demonstrates that the method, called "CREANN" (Causal Relationship Estimation by Artificial Neural Network), can estimate time-invariant and time-varying effective connectivity in terms of MVAR coefficients. The method shows robustness with respect to noise level of data. Furthermore, the estimations are not significantly influenced by the model order (considered time-lag), and the different initial conditions (initial random weights and parameters of the network). CREANN is also applied to EEG data collected during a memory recognition task. The results implicate that it can show changes in the information flow between brain regions, involving in the episodic memory retrieval process. These convincing results emphasize that CREANN can be used as an appropriate method to estimate the causal relationship among brain signals.

Information flow in layered networks of non-monotonic units

NASA Astrophysics Data System (ADS)

Schittler Neves, Fabio; Martim Schubert, Benno; Erichsen, Rubem, Jr.

2015-07-01

Layered neural networks are feedforward structures that yield robust parallel and distributed pattern recognition. Even though much attention has been paid to pattern retrieval properties in such systems, many aspects of their dynamics are not yet well characterized or understood. In this work we study, at different temperatures, the memory activity and information flows through layered networks in which the elements are the simplest binary odd non-monotonic function. Our results show that, considering a standard Hebbian learning approach, the network information content has its maximum always at the monotonic limit, even though the maximum memory capacity can be found at non-monotonic values for small enough temperatures. Furthermore, we show that such systems exhibit rich macroscopic dynamics, including not only fixed point solutions of its iterative map, but also cyclic and chaotic attractors that also carry information.

Multi-layer holographic bifurcative neural network system for real-time adaptive EOS data analysis

NASA Technical Reports Server (NTRS)

Liu, Hua-Kuang; Huang, K. S.; Diep, J.

1993-01-01

Optical data processing techniques have the inherent advantage of high data throughout, low weight and low power requirements. These features are particularly desirable for onboard spacecraft in-situ real-time data analysis and data compression applications. the proposed multi-layer optical holographic neural net pattern recognition technique will utilize the nonlinear photorefractive devices for real-time adaptive learning to classify input data content and recognize unexpected features. Information can be stored either in analog or digital form in a nonlinear photofractive device. The recording can be accomplished in time scales ranging from milliseconds to microseconds. When a system consisting of these devices is organized in a multi-layer structure, a feedforward neural net with bifurcating data classification capability is formed. The interdisciplinary research will involve the collaboration with top digital computer architecture experts at the University of Southern California.

Psychological Processing in Chronic Pain: A Neural Systems Approach

PubMed Central

Simons, Laura; Elman, Igor; Borsook, David

2014-01-01

Our understanding of chronic pain involves complex brain circuits that include sensory, emotional, cognitive and interoceptive processing. The feed-forward interactions between physical (e.g., trauma) and emotional pain and the consequences of altered psychological status on the expression of pain have made the evaluation and treatment of chronic pain a challenge in the clinic. By understanding the neural circuits involved in psychological processes, a mechanistic approach to the implementation of psychology-based treatments may be better understood. In this review we evaluate some of the principle processes that may be altered as a consequence of chronic pain in the context of localized and integrated neural networks. These changes are ongoing, vary in their magnitude, and their hierarchical manifestations, and may be temporally and sequentially altered by treatments, and all contribute to an overall pain phenotype. Furthermore, we link altered psychological processes to specific evidence-based treatments to put forth a model of pain neuroscience psychology. PMID:24374383

Detection of Pigment Networks in Dermoscopy Images

NASA Astrophysics Data System (ADS)

Eltayef, Khalid; Li, Yongmin; Liu, Xiaohui

2017-02-01

One of the most important structures in dermoscopy images is the pigment network, which is also one of the most challenging and fundamental task for dermatologists in early detection of melanoma. This paper presents an automatic system to detect pigment network from dermoscopy images. The design of the proposed algorithm consists of four stages. First, a pre-processing algorithm is carried out in order to remove the noise and improve the quality of the image. Second, a bank of directional filters and morphological connected component analysis are applied to detect the pigment networks. Third, features are extracted from the detected image, which can be used in the subsequent stage. Fourth, the classification process is performed by applying feed-forward neural network, in order to classify the region as either normal or abnormal skin. The method was tested on a dataset of 200 dermoscopy images from Hospital Pedro Hispano (Matosinhos), and better results were produced compared to previous studies.

Artificial neural networks in knee injury risk evaluation among professional football players

NASA Astrophysics Data System (ADS)

Martyna, Michałowska; Tomasz, Walczak; Krzysztof, Grabski Jakub; Monika, Grygorowicz

2018-01-01

Lower limb injury risk assessment was proposed, based on isokinetic examination that is a part of standard athlete's biomechanical evaluation performed mainly twice a year. Information about non-contact knee injury (or lack of the injury) sustained within twelve months after isokinetic test, confirmed in USG were verified. Three the most common types of football injuries were taken into consideration: anterior cruciate ligament (ACL) rupture, hamstring and quadriceps muscles injuries. 22 parameters, obtained from isokinetic tests were divided into 4 groups and used as input parameters of five feedforward artificial neural networks (ANNs). The 5th group consisted of all considered parameters. The networks were trained with the use of Levenberg-Marquardt backpropagation algorithm to return value close to 1 for the sets of parameters corresponding injury event and close to 0 for parameters with no injury recorded within 6 - 12 months after isokinetic test. Results of this study shows that ANN might be useful tools, which simplify process of simultaneous interpretation of many numerical parameters, but the most important factor that significantly influence the results is database used for ANN training.

Predicting high-risk preterm birth using artificial neural networks.

PubMed

Catley, Christina; Frize, Monique; Walker, C Robin; Petriu, Dorina C

2006-07-01

A reengineered approach to the early prediction of preterm birth is presented as a complimentary technique to the current procedure of using costly and invasive clinical testing on high-risk maternal populations. Artificial neural networks (ANNs) are employed as a screening tool for preterm birth on a heterogeneous maternal population; risk estimations use obstetrical variables available to physicians before 23 weeks gestation. The objective was to assess if ANNs have a potential use in obstetrical outcome estimations in low-risk maternal populations. The back-propagation feedforward ANN was trained and tested on cases with eight input variables describing the patient's obstetrical history; the output variables were: 1) preterm birth; 2) high-risk preterm birth; and 3) a refined high-risk preterm birth outcome excluding all cases where resuscitation was delivered in the form of free flow oxygen. Artificial training sets were created to increase the distribution of the underrepresented class to 20%. Training on the refined high-risk preterm birth model increased the network's sensitivity to 54.8%, compared to just over 20% for the nonartificially distributed preterm birth model.

Neural attractor network for application in visual field data classification.

PubMed

Fink, Wolfgang

2004-07-07

The purpose was to introduce a novel method for computer-based classification of visual field data derived from perimetric examination, that may act as a 'counsellor', providing an independent 'second opinion' to the diagnosing physician. The classification system consists of a Hopfield-type neural attractor network that obtains its input data from perimetric examination results. An iterative relaxation process determines the states of the neurons dynamically. Therefore, even 'noisy' perimetric output, e.g., early stages of a disease, may eventually be classified correctly according to the predefined idealized visual field defect (scotoma) patterns, stored as attractors of the network, that are found with diseases of the eye, optic nerve and the central nervous system. Preliminary tests of the classification system on real visual field data derived from perimetric examinations have shown a classification success of over 80%. Some of the main advantages of the Hopfield-attractor-network-based approach over feed-forward type neural networks are: (1) network architecture is defined by the classification problem; (2) no training is required to determine the neural coupling strengths; (3) assignment of an auto-diagnosis confidence level is possible by means of an overlap parameter and the Hamming distance. In conclusion, the novel method for computer-based classification of visual field data, presented here, furnishes a valuable first overview and an independent 'second opinion' in judging perimetric examination results, pointing towards a final diagnosis by a physician. It should not be considered a substitute for the diagnosing physician. Thanks to the worldwide accessibility of the Internet, the classification system offers a promising perspective towards modern computer-assisted diagnosis in both medicine and tele-medicine, for example and in particular, with respect to non-ophthalmic clinics or in communities where perimetric expertise is not readily available.

Generation of optimal artificial neural networks using a pattern search algorithm: application to approximation of chemical systems.

PubMed

Ihme, Matthias; Marsden, Alison L; Pitsch, Heinz

2008-02-01

A pattern search optimization method is applied to the generation of optimal artificial neural networks (ANNs). Optimization is performed using a mixed variable extension to the generalized pattern search method. This method offers the advantage that categorical variables, such as neural transfer functions and nodal connectivities, can be used as parameters in optimization. When used together with a surrogate, the resulting algorithm is highly efficient for expensive objective functions. Results demonstrate the effectiveness of this method in optimizing an ANN for the number of neurons, the type of transfer function, and the connectivity among neurons. The optimization method is applied to a chemistry approximation of practical relevance. In this application, temperature and a chemical source term are approximated as functions of two independent parameters using optimal ANNs. Comparison of the performance of optimal ANNs with conventional tabulation methods demonstrates equivalent accuracy by considerable savings in memory storage. The architecture of the optimal ANN for the approximation of the chemical source term consists of a fully connected feedforward network having four nonlinear hidden layers and 117 synaptic weights. An equivalent representation of the chemical source term using tabulation techniques would require a 500 x 500 grid point discretization of the parameter space.

Quantification of whey in fluid milk using confocal Raman microscopy and artificial neural network.

PubMed

Alves da Rocha, Roney; Paiva, Igor Moura; Anjos, Virgílio; Furtado, Marco Antônio Moreira; Bell, Maria José Valenzuela

2015-06-01

In this work, we assessed the use of confocal Raman microscopy and artificial neural network as a practical method to assess and quantify adulteration of fluid milk by addition of whey. Milk samples with added whey (from 0 to 100%) were prepared, simulating different levels of fraudulent adulteration. All analyses were carried out by direct inspection at the light microscope after depositing drops from each sample on a microscope slide and drying them at room temperature. No pre- or posttreatment (e.g., sample preparation or spectral correction) was required in the analyses. Quantitative determination of adulteration was performed through a feed-forward artificial neural network (ANN). Different ANN configurations were evaluated based on their coefficient of determination (R2) and root mean square error values, which were criteria for selecting the best predictor model. In the selected model, we observed that data from both training and validation subsets presented R2>99.99%, indicating that the combination of confocal Raman microscopy and ANN is a rapid, simple, and efficient method to quantify milk adulteration by whey. Because sample preparation and postprocessing of spectra were not required, the method has potential applications in health surveillance and food quality monitoring. Copyright © 2015 American Dairy Science Association. Published by Elsevier Inc. All rights reserved.

On the use of harmony search algorithm in the training of wavelet neural networks

NASA Astrophysics Data System (ADS)

Lai, Kee Huong; Zainuddin, Zarita; Ong, Pauline

2015-10-01

Wavelet neural networks (WNNs) are a class of feedforward neural networks that have been used in a wide range of industrial and engineering applications to model the complex relationships between the given inputs and outputs. The training of WNNs involves the configuration of the weight values between neurons. The backpropagation training algorithm, which is a gradient-descent method, can be used for this training purpose. Nonetheless, the solutions found by this algorithm often get trapped at local minima. In this paper, a harmony search-based algorithm is proposed for the training of WNNs. The training of WNNs, thus can be formulated as a continuous optimization problem, where the objective is to maximize the overall classification accuracy. Each candidate solution proposed by the harmony search algorithm represents a specific WNN architecture. In order to speed up the training process, the solution space is divided into disjoint partitions during the random initialization step of harmony search algorithm. The proposed training algorithm is tested onthree benchmark problems from the UCI machine learning repository, as well as one real life application, namely, the classification of electroencephalography signals in the task of epileptic seizure detection. The results obtained show that the proposed algorithm outperforms the traditional harmony search algorithm in terms of overall classification accuracy.

Modeling the Effects of Cu Content and Deformation Variables on the High-Temperature Flow Behavior of Dilute Al-Fe-Si Alloys Using an Artificial Neural Network.

PubMed

Shakiba, Mohammad; Parson, Nick; Chen, X-Grant

2016-06-30

The hot deformation behavior of Al-0.12Fe-0.1Si alloys with varied amounts of Cu (0.002-0.31 wt %) was investigated by uniaxial compression tests conducted at different temperatures (400 °C-550 °C) and strain rates (0.01-10 s -1 ). The results demonstrated that flow stress decreased with increasing deformation temperature and decreasing strain rate, while flow stress increased with increasing Cu content for all deformation conditions studied due to the solute drag effect. Based on the experimental data, an artificial neural network (ANN) model was developed to study the relationship between chemical composition, deformation variables and high-temperature flow behavior. A three-layer feed-forward back-propagation artificial neural network with 20 neurons in a hidden layer was established in this study. The input parameters were Cu content, temperature, strain rate and strain, while the flow stress was the output. The performance of the proposed model was evaluated using the K-fold cross-validation method. The results showed excellent generalization capability of the developed model. Sensitivity analysis indicated that the strain rate is the most important parameter, while the Cu content exhibited a modest but significant influence on the flow stress.

Application of adaptive boosting to EP-derived multilayer feed-forward neural networks (MLFN) to improve benign/malignant breast cancer classification

NASA Astrophysics Data System (ADS)

Land, Walker H., Jr.; Masters, Timothy D.; Lo, Joseph Y.; McKee, Dan

2001-07-01

A new neural network technology was developed for improving the benign/malignant diagnosis of breast cancer using mammogram findings. A new paradigm, Adaptive Boosting (AB), uses a markedly different theory in solutioning Computational Intelligence (CI) problems. AB, a new machine learning paradigm, focuses on finding weak learning algorithm(s) that initially need to provide slightly better than random performance (i.e., approximately 55%) when processing a mammogram training set. Then, by successive development of additional architectures (using the mammogram training set), the adaptive boosting process improves the performance of the basic Evolutionary Programming derived neural network architectures. The results of these several EP-derived hybrid architectures are then intelligently combined and tested using a similar validation mammogram data set. Optimization focused on improving specificity and positive predictive value at very high sensitivities, where an analysis of the performance of the hybrid would be most meaningful. Using the DUKE mammogram database of 500 biopsy proven samples, on average this hybrid was able to achieve (under statistical 5-fold cross-validation) a specificity of 48.3% and a positive predictive value (PPV) of 51.8% while maintaining 100% sensitivity. At 97% sensitivity, a specificity of 56.6% and a PPV of 55.8% were obtained.

Modeling the Effects of Cu Content and Deformation Variables on the High-Temperature Flow Behavior of Dilute Al-Fe-Si Alloys Using an Artificial Neural Network

PubMed Central

Shakiba, Mohammad; Parson, Nick; Chen, X.-Grant

2016-01-01

The hot deformation behavior of Al-0.12Fe-0.1Si alloys with varied amounts of Cu (0.002–0.31 wt %) was investigated by uniaxial compression tests conducted at different temperatures (400 °C–550 °C) and strain rates (0.01–10 s−1). The results demonstrated that flow stress decreased with increasing deformation temperature and decreasing strain rate, while flow stress increased with increasing Cu content for all deformation conditions studied due to the solute drag effect. Based on the experimental data, an artificial neural network (ANN) model was developed to study the relationship between chemical composition, deformation variables and high-temperature flow behavior. A three-layer feed-forward back-propagation artificial neural network with 20 neurons in a hidden layer was established in this study. The input parameters were Cu content, temperature, strain rate and strain, while the flow stress was the output. The performance of the proposed model was evaluated using the K-fold cross-validation method. The results showed excellent generalization capability of the developed model. Sensitivity analysis indicated that the strain rate is the most important parameter, while the Cu content exhibited a modest but significant influence on the flow stress. PMID:28773658

A Novel Extreme Learning Machine Classification Model for e-Nose Application Based on the Multiple Kernel Approach

PubMed Central

Jian, Yulin; Huang, Daoyu; Yan, Jia; Lu, Kun; Huang, Ying; Wen, Tailai; Zeng, Tanyue; Zhong, Shijie; Xie, Qilong

2017-01-01

A novel classification model, named the quantum-behaved particle swarm optimization (QPSO)-based weighted multiple kernel extreme learning machine (QWMK-ELM), is proposed in this paper. Experimental validation is carried out with two different electronic nose (e-nose) datasets. Being different from the existing multiple kernel extreme learning machine (MK-ELM) algorithms, the combination coefficients of base kernels are regarded as external parameters of single-hidden layer feedforward neural networks (SLFNs). The combination coefficients of base kernels, the model parameters of each base kernel, and the regularization parameter are optimized by QPSO simultaneously before implementing the kernel extreme learning machine (KELM) with the composite kernel function. Four types of common single kernel functions (Gaussian kernel, polynomial kernel, sigmoid kernel, and wavelet kernel) are utilized to constitute different composite kernel functions. Moreover, the method is also compared with other existing classification methods: extreme learning machine (ELM), kernel extreme learning machine (KELM), k-nearest neighbors (KNN), support vector machine (SVM), multi-layer perceptron (MLP), radical basis function neural network (RBFNN), and probabilistic neural network (PNN). The results have demonstrated that the proposed QWMK-ELM outperforms the aforementioned methods, not only in precision, but also in efficiency for gas classification. PMID:28629202

Stabilization of burn conditions in a thermonuclear reactor using artificial neural networks

NASA Astrophysics Data System (ADS)

Vitela, Javier E.; Martinell, Julio J.

1998-02-01

In this work we develop an artificial neural network (ANN) for the feedback stabilization of a thermonuclear reactor at nearly ignited burn conditions. A volume-averaged zero-dimensional nonlinear model is used to represent the time evolution of the electron density, the relative density of alpha particles and the temperature of the plasma, where a particular scaling law for the energy confinement time previously used by other authors, was adopted. The control actions include the concurrent modulation of the D-T refuelling rate, the injection of a neutral He-4 beam and an auxiliary heating power modulation, which are constrained to take values within a maximum and minimum levels. For this purpose a feedforward multilayer artificial neural network with sigmoidal activation function is trained using a back-propagation through-time technique. Numerical examples are used to illustrate the behaviour of the resulting ANN-dynamical system configuration. It is concluded that the resulting ANN can successfully stabilize the nonlinear model of the thermonuclear reactor at nearly ignited conditions for temperature and density departures significantly far from their nominal operating values. The NN-dynamical system configuration is shown to be robust with respect to the thermalization time of the alpha particles for perturbations within the region used to train the NN.

Accelerated Training for Large Feedforward Neural Networks

NASA Technical Reports Server (NTRS)

Stepniewski, Slawomir W.; Jorgensen, Charles C.

1998-01-01

In this paper we introduce a new training algorithm, the scaled variable metric (SVM) method. Our approach attempts to increase the convergence rate of the modified variable metric method. It is also combined with the RBackprop algorithm, which computes the product of the matrix of second derivatives (Hessian) with an arbitrary vector. The RBackprop method allows us to avoid computationally expensive, direct line searches. In addition, it can be utilized in the new, 'predictive' updating technique of the inverse Hessian approximation. We have used directional slope testing to adjust the step size and found that this strategy works exceptionally well in conjunction with the Rbackprop algorithm. Some supplementary, but nevertheless important enhancements to the basic training scheme such as improved setting of a scaling factor for the variable metric update and computationally more efficient procedure for updating the inverse Hessian approximation are presented as well. We summarize by comparing the SVM method with four first- and second- order optimization algorithms including a very effective implementation of the Levenberg-Marquardt method. Our tests indicate promising computational speed gains of the new training technique, particularly for large feedforward networks, i.e., for problems where the training process may be the most laborious.

Feedforward Categorization on AER Motion Events Using Cortex-Like Features in a Spiking Neural Network.

PubMed

Zhao, Bo; Ding, Ruoxi; Chen, Shoushun; Linares-Barranco, Bernabe; Tang, Huajin

2015-09-01

This paper introduces an event-driven feedforward categorization system, which takes data from a temporal contrast address event representation (AER) sensor. The proposed system extracts bio-inspired cortex-like features and discriminates different patterns using an AER based tempotron classifier (a network of leaky integrate-and-fire spiking neurons). One of the system's most appealing characteristics is its event-driven processing, with both input and features taking the form of address events (spikes). The system was evaluated on an AER posture dataset and compared with two recently developed bio-inspired models. Experimental results have shown that it consumes much less simulation time while still maintaining comparable performance. In addition, experiments on the Mixed National Institute of Standards and Technology (MNIST) image dataset have demonstrated that the proposed system can work not only on raw AER data but also on images (with a preprocessing step to convert images into AER events) and that it can maintain competitive accuracy even when noise is added. The system was further evaluated on the MNIST dynamic vision sensor dataset (in which data is recorded using an AER dynamic vision sensor), with testing accuracy of 88.14%.

Manifold absolute pressure estimation using neural network with hybrid training algorithm

PubMed Central

Selamat, Hazlina; Alimin, Ahmad Jais; Haniff, Mohamad Fadzli

2017-01-01

In a modern small gasoline engine fuel injection system, the load of the engine is estimated based on the measurement of the manifold absolute pressure (MAP) sensor, which took place in the intake manifold. This paper present a more economical approach on estimating the MAP by using only the measurements of the throttle position and engine speed, resulting in lower implementation cost. The estimation was done via two-stage multilayer feed-forward neural network by combining Levenberg-Marquardt (LM) algorithm, Bayesian Regularization (BR) algorithm and Particle Swarm Optimization (PSO) algorithm. Based on the results found in 20 runs, the second variant of the hybrid algorithm yields a better network performance than the first variant of hybrid algorithm, LM, LM with BR and PSO by estimating the MAP closely to the simulated MAP values. By using a valid experimental training data, the estimator network that trained with the second variant of the hybrid algorithm showed the best performance among other algorithms when used in an actual retrofit fuel injection system (RFIS). The performance of the estimator was also validated in steady-state and transient condition by showing a closer MAP estimation to the actual value. PMID:29190779

Spatiotemporal neural network dynamics for the processing of dynamic facial expressions.

PubMed

Sato, Wataru; Kochiyama, Takanori; Uono, Shota

2015-07-24

The dynamic facial expressions of emotion automatically elicit multifaceted psychological activities; however, the temporal profiles and dynamic interaction patterns of brain activities remain unknown. We investigated these issues using magnetoencephalography. Participants passively observed dynamic facial expressions of fear and happiness, or dynamic mosaics. Source-reconstruction analyses utilizing functional magnetic-resonance imaging data revealed higher activation in broad regions of the bilateral occipital and temporal cortices in response to dynamic facial expressions than in response to dynamic mosaics at 150-200 ms and some later time points. The right inferior frontal gyrus exhibited higher activity for dynamic faces versus mosaics at 300-350 ms. Dynamic causal-modeling analyses revealed that dynamic faces activated the dual visual routes and visual-motor route. Superior influences of feedforward and feedback connections were identified before and after 200 ms, respectively. These results indicate that hierarchical, bidirectional neural network dynamics within a few hundred milliseconds implement the processing of dynamic facial expressions.

A Telescopic Binary Learning Machine for Training Neural Networks.

PubMed

Brunato, Mauro; Battiti, Roberto

2017-03-01

This paper proposes a new algorithm based on multiscale stochastic local search with binary representation for training neural networks [binary learning machine (BLM)]. We study the effects of neighborhood evaluation strategies, the effect of the number of bits per weight and that of the maximum weight range used for mapping binary strings to real values. Following this preliminary investigation, we propose a telescopic multiscale version of local search, where the number of bits is increased in an adaptive manner, leading to a faster search and to local minima of better quality. An analysis related to adapting the number of bits in a dynamic way is presented. The control on the number of bits, which happens in a natural manner in the proposed method, is effective to increase the generalization performance. The learning dynamics are discussed and validated on a highly nonlinear artificial problem and on real-world tasks in many application domains; BLM is finally applied to a problem requiring either feedforward or recurrent architectures for feedback control.

Spatiotemporal neural network dynamics for the processing of dynamic facial expressions

PubMed Central

Sato, Wataru; Kochiyama, Takanori; Uono, Shota

2015-01-01

The dynamic facial expressions of emotion automatically elicit multifaceted psychological activities; however, the temporal profiles and dynamic interaction patterns of brain activities remain unknown. We investigated these issues using magnetoencephalography. Participants passively observed dynamic facial expressions of fear and happiness, or dynamic mosaics. Source-reconstruction analyses utilizing functional magnetic-resonance imaging data revealed higher activation in broad regions of the bilateral occipital and temporal cortices in response to dynamic facial expressions than in response to dynamic mosaics at 150–200 ms and some later time points. The right inferior frontal gyrus exhibited higher activity for dynamic faces versus mosaics at 300–350 ms. Dynamic causal-modeling analyses revealed that dynamic faces activated the dual visual routes and visual–motor route. Superior influences of feedforward and feedback connections were identified before and after 200 ms, respectively. These results indicate that hierarchical, bidirectional neural network dynamics within a few hundred milliseconds implement the processing of dynamic facial expressions. PMID:26206708

Comparison of different modelling approaches of drive train temperature for the purposes of wind turbine failure detection

NASA Astrophysics Data System (ADS)

Tautz-Weinert, J.; Watson, S. J.

2016-09-01

Effective condition monitoring techniques for wind turbines are needed to improve maintenance processes and reduce operational costs. Normal behaviour modelling of temperatures with information from other sensors can help to detect wear processes in drive trains. In a case study, modelling of bearing and generator temperatures is investigated with operational data from the SCADA systems of more than 100 turbines. The focus is here on automated training and testing on a farm level to enable an on-line system, which will detect failures without human interpretation. Modelling based on linear combinations, artificial neural networks, adaptive neuro-fuzzy inference systems, support vector machines and Gaussian process regression is compared. The selection of suitable modelling inputs is discussed with cross-correlation analyses and a sensitivity study, which reveals that the investigated modelling techniques react in different ways to an increased number of inputs. The case study highlights advantages of modelling with linear combinations and artificial neural networks in a feedforward configuration.

Automated electronic tongue based on potentiometric sensors for the determination of a trinary anionic surfactant mixture.

PubMed

Cortina, Montserrat; Ecker, Christina; Calvo, Daniel; del Valle, Manuel

2008-01-22

An automated electronic tongue consisting of an array of potentiometric sensors and an artificial neural network (ANN) has been developed to resolve mixtures of anionic surfactants. The sensor array was formed by five different flow-through sensors for anionic surfactants, based on poly(vinyl chloride) membranes having cross-sensitivity features. Feedforward multilayer neural networks were used to predict surfactant concentrations. As a great amount of information is required for the correct modelling of the sensors response, a sequential injection analysis (SIA) system was used to automatically provide it. Dodecylsulfate (DS(-)), dodecylbenzenesulfonate (DBS(-)) and alpha-alkene sulfonate (ALF(-)) formed the three-analyte study case resolved in this work. Their concentrations varied from 0.2 to 4mM for ALF(-) and DBS(-) and from 0.2 to 5mM for DS(-). Good prediction ability was obtained with correlation coefficients better than 0.933 when the obtained values were compared with those expected for a set of 16 external test samples not used for training.

Model risk for European-style stock index options.

PubMed

Gençay, Ramazan; Gibson, Rajna

2007-01-01

In empirical modeling, there have been two strands for pricing in the options literature, namely the parametric and nonparametric models. Often, the support for the nonparametric methods is based on a benchmark such as the Black-Scholes (BS) model with constant volatility. In this paper, we study the stochastic volatility (SV) and stochastic volatility random jump (SVJ) models as parametric benchmarks against feedforward neural network (FNN) models, a class of neural network models. Our choice for FNN models is due to their well-studied universal approximation properties of an unknown function and its partial derivatives. Since the partial derivatives of an option pricing formula are risk pricing tools, an accurate estimation of the unknown option pricing function is essential for pricing and hedging. Our findings indicate that FNN models offer themselves as robust option pricing tools, over their sophisticated parametric counterparts in predictive settings. There are two routes to explain the superiority of FNN models over the parametric models in forecast settings. These are nonnormality of return distributions and adaptive learning.

A new simple /spl infin/OH neuron model as a biologically plausible principal component analyzer.

PubMed

Jankovic, M V

2003-01-01

A new approach to unsupervised learning in a single-layer neural network is discussed. An algorithm for unsupervised learning based upon the Hebbian learning rule is presented. A simple neuron model is analyzed. A dynamic neural model, which contains both feed-forward and feedback connections between the input and the output, has been adopted. The, proposed learning algorithm could be more correctly named self-supervised rather than unsupervised. The solution proposed here is a modified Hebbian rule, in which the modification of the synaptic strength is proportional not to pre- and postsynaptic activity, but instead to the presynaptic and averaged value of postsynaptic activity. It is shown that the model neuron tends to extract the principal component from a stationary input vector sequence. Usually accepted additional decaying terms for the stabilization of the original Hebbian rule are avoided. Implementation of the basic Hebbian scheme would not lead to unrealistic growth of the synaptic strengths, thanks to the adopted network structure.

Neural network-based position synchronised internal force control scheme for cooperative manipulator system

NASA Astrophysics Data System (ADS)

Wang, Jin; Xu, Fan; Lu, GuoDong

2017-09-01

More complex problems of simultaneous position and internal force control occur with cooperative manipulator systems than that of a single one. In the presence of unwanted parametric and modelling uncertainties as well as external disturbances, a decentralised position synchronised force control scheme is proposed. With a feedforward neural network estimating engine, a precise model of the system dynamics is not required. Unlike conventional cooperative or synchronised controllers, virtual position and virtual synchronisation errors are introduced for internal force tracking control and task space position synchronisation. Meanwhile joint space synchronisation and force measurement are unnecessary. Together with simulation studies and analysis, the position and the internal force errors are shown to asymptotically converge to zero. Moreover, the controller exhibits different characteristics with selected synchronisation factors. Under certain settings, it can deal with temporary cooperation by an intelligent retreat mechanism, where less internal force would occur and rigid collision can be avoided. Using a Lyapunov stability approach, the controller is proven to be robust in face of the aforementioned uncertainties.

Robust artificial intelligence tool for automatic start-up of the supplementary medium feeding in recombinant E. coli cultivations.

PubMed

Horta, Antônio Carlos Luperni; da Silva, Adilson José; Sargo, Cíntia Regina; Gonçalves, Viviane Maimoni; Zangirolami, Teresa Cristina; Giordano, Roberto de Campos

2011-09-01

One of the most important events in fed-batch fermentations is the definition of the moment to start the feeding. This paper presents a methodology for a rational selection of the architecture of an artificial intelligence (AI) system, based on a neural network committee (NNC), which identifies the end of the batch phase. The AI system was successfully used during high cell density cultivations of recombinant Escherichia coli. The AI algorithm was validated for different systems, expressing three antigens to be used in human and animal vaccines: fragments of surface proteins of Streptococcus pneumoniae (PspA), clades 1 and 3, and of Erysipelothrix rhusiopathiae (SpaA). Standard feed-forward neural networks (NNs), with a single hidden layer, were the basis for the NNC. The NN architecture with best performance had the following inputs: stirrer speed, inlet air, and oxygen flow rates, carbon dioxide evolution rate, and CO2 molar fraction in the exhaust gas.

Feedforward Compensation Mediated by the Central and Peripheral Actions of a Single Neuropeptide Discovered Using Representational Difference Analysis

PubMed Central

Jing, Jian; Sweedler, Jonathan V; Cropper, Elizabeth C; Alexeeva, Vera; Park, Ji-Ho; Romanova, Elena V.; Xie, Fang; Dembrow, Nikolai C.; Ludwar, Bjoern C.; Weiss, Klaudiusz R; Vilim, Ferdinand S

2010-01-01

Compensatory mechanisms are often used to achieve stability by reducing variance, which can be accomplished via negative feedback during homeostatic regulation. In principle, compensation can also be implemented through feedforward mechanisms where a regulator acts to offset the anticipated output variation; however, few such neural mechanisms have been demonstrated. We provide evidence that an Aplysia neuropeptide, identified using an enhanced representational difference analysis procedure, implements feedforward compensation within the feeding network. We named the novel peptide allatotropin-related peptide (ATRP) because of its similarity to insect allatotropin. Mass spectrometry confirmed the peptide's identity, and in situ hybridization and immunostaining mapped its distribution in the Aplysia CNS. ATRP is present in the higher-order cerebral-buccal interneuron (CBI), CBI-4, but not in CBI-2. Previous work showed that CBI-4-elicited motor programs have a shorter protraction duration than those elicited by CBI-2. Here we show that ATRP shortens protraction duration of CBI-2-elicited ingestive programs, suggesting a contribution of ATRP to the parametric differences between CBI-4- and CBI-2-evoked programs. Importantly, because Aplysia muscle contractions are a graded function of motoneuronal activity, one consequence of the shortening of protraction is that it can weaken protraction movements. However, this potential weakening is offset by feedforward compensatory actions exerted by ATRP. Centrally, ATRP increases the activity of protraction motoneurons. Moreover, ATRP is present in peripheral varicosities of protraction motoneurons and enhances peripheral motoneuron-elicited protraction muscle contractions. Therefore, feedforward compensatory mechanisms mediated by ATRP make it possible to generate a faster movement with an amplitude that is not greatly reduced, thereby producing stability. PMID:21147994

Nature-Inspired Cognitive Evolution to Play MS. Pac-Man

NASA Astrophysics Data System (ADS)

Tan, Tse Guan; Teo, Jason; Anthony, Patricia

Recent developments in nature-inspired computation have heightened the need for research into the three main areas of scientific, engineering and industrial applications. Some approaches have reported that it is able to solve dynamic problems and very useful for improving the performance of various complex systems. So far however, there has been little discussion about the effectiveness of the application of these models to computer and video games in particular. The focus of this research is to explore the hybridization of nature-inspired computation methods for optimization of neural network-based cognition in video games, in this case the combination of a neural network with an evolutionary algorithm. In essence, a neural network is an attempt to mimic the extremely complex human brain system, which is building an artificial brain that is able to self-learn intelligently. On the other hand, an evolutionary algorithm is to simulate the biological evolutionary processes that evolve potential solutions in order to solve the problems or tasks by applying the genetic operators such as crossover, mutation and selection into the solutions. This paper investigates the abilities of Evolution Strategies (ES) to evolve feed-forward artificial neural network's internal parameters (i.e. weight and bias values) for automatically generating Ms. Pac-man controllers. The main objective of this game is to clear a maze of dots while avoiding the ghosts and to achieve the highest possible score. The experimental results show that an ES-based system can be successfully applied to automatically generate artificial intelligence for a complex, dynamic and highly stochastic video game environment.

CROSS-DISCIPLINARY PHYSICS AND RELATED AREAS OF SCIENCE AND TECHNOLOGY: Mechanism for propagation of rate signals through a 10-layer feedforward neuronal network

NASA Astrophysics Data System (ADS)

Li, Jie; Yu, Wan-Qing; Xu, Ding; Liu, Feng; Wang, Wei

2009-12-01

Using numerical simulations, we explore the mechanism for propagation of rate signals through a 10-layer feedforward network composed of Hodgkin-Huxley (HH) neurons with sparse connectivity. When white noise is afferent to the input layer, neuronal firing becomes progressively more synchronous in successive layers and synchrony is well developed in deeper layers owing to the feedforward connections between neighboring layers. The synchrony ensures the successful propagation of rate signals through the network when the synaptic conductance is weak. As the synaptic time constant τsyn varies, coherence resonance is observed in the network activity due to the intrinsic property of HH neurons. This makes the output firing rate single-peaked as a function of τsyn, suggesting that the signal propagation can be modulated by the synaptic time constant. These results are consistent with experimental results and advance our understanding of how information is processed in feedforward networks.

Use of a Neural Net to Model the Impact of Optical Coherence Tomography Abnormalities on Vision in Age-related Macular Degeneration.

PubMed

Aslam, Tariq M; Zaki, Haider R; Mahmood, Sajjad; Ali, Zaria C; Ahmad, Nur A; Thorell, Mariana R; Balaskas, Konstantinos

2018-01-01

To develop a neural network for the estimation of visual acuity from optical coherence tomography (OCT) images of patients with neovascular age-related macular degeneration (AMD) and to demonstrate its use to model the impact of specific controlled OCT changes on vision. Artificial intelligence (neural network) study. We assessed 1400 OCT scans of patients with neovascular AMD. Fifteen physical features for each eligible OCT, as well as patient age, were used as input data and corresponding recorded visual acuity as the target data to train, validate, and test a supervised neural network. We then applied this network to model the impact on acuity of defined OCT changes in subretinal fluid, subretinal hyperreflective material, and loss of external limiting membrane (ELM) integrity. A total of 1210 eligible OCT scans were analyzed, resulting in 1210 data points, which were each 16-dimensional. A 10-layer feed-forward neural network with 1 hidden layer of 10 neurons was trained to predict acuity and demonstrated a root mean square error of 8.2 letters for predicted compared to actual visual acuity and a mean regression coefficient of 0.85. A virtual model using this network demonstrated the relationship of visual acuity to specific, programmed changes in OCT characteristics. When ELM is intact, there is a shallow decline in acuity with increasing subretinal fluid but a much steeper decline with equivalent increasing subretinal hyperreflective material. When ELM is not intact, all visual acuities are reduced. Increasing subretinal hyperreflective material or subretinal fluid in this circumstance reduces vision further still, but with a smaller gradient than when ELM is intact. The supervised machine learning neural network developed is able to generate an estimated visual acuity value from OCT images in a population of patients with AMD. These findings should be of clinical and research interest in macular degeneration, for example in estimating visual prognosis or highlighting the importance of developing treatments targeting more visually destructive pathologies. Copyright © 2017 Elsevier Inc. All rights reserved.

Network architectures and circuit function: testing alternative hypotheses in multifunctional networks.

PubMed

Leonard, J L

2000-05-01

Understanding how species-typical movement patterns are organized in the nervous system is a central question in neurobiology. The current explanations involve 'alphabet' models in which an individual neuron may participate in the circuit for several behaviors but each behavior is specified by a specific neural circuit. However, not all of the well-studied model systems fit the 'alphabet' model. The 'equation' model provides an alternative possibility, whereby a system of parallel motor neurons, each with a unique (but overlapping) field of innervation, can account for the production of stereotyped behavior patterns by variable circuits. That is, it is possible for such patterns to arise as emergent properties of a generalized neural network in the absence of feedback, a simple version of a 'self-organizing' behavioral system. Comparison of systems of identified neurons suggest that the 'alphabet' model may account for most observations where CPGs act to organize motor patterns. Other well-known model systems, involving architectures corresponding to feed-forward neural networks with a hidden layer, may organize patterned behavior in a manner consistent with the 'equation' model. Such architectures are found in the Mauthner and reticulospinal circuits, 'escape' locomotion in cockroaches, CNS control of Aplysia gill, and may also be important in the coordination of sensory information and motor systems in insect mushroom bodies and the vertebrate hippocampus. The hidden layer of such networks may serve as an 'internal representation' of the behavioral state and/or body position of the animal, allowing the animal to fine-tune oriented, or particularly context-sensitive, movements to the prevalent conditions. Experiments designed to distinguish between the two models in cases where they make mutually exclusive predictions provide an opportunity to elucidate the neural mechanisms by which behavior is organized in vivo and in vitro. Copyright 2000 S. Karger AG, Basel

Ab initio potential-energy surfaces for complex, multichannel systems using modified novelty sampling and feedforward neural networks

NASA Astrophysics Data System (ADS)

Raff, L. M.; Malshe, M.; Hagan, M.; Doughan, D. I.; Rockley, M. G.; Komanduri, R.

2005-02-01

A neural network/trajectory approach is presented for the development of accurate potential-energy hypersurfaces that can be utilized to conduct ab initio molecular dynamics (AIMD) and Monte Carlo studies of gas-phase chemical reactions, nanometric cutting, and nanotribology, and of a variety of mechanical properties of importance in potential microelectromechanical systems applications. The method is sufficiently robust that it can be applied to a wide range of polyatomic systems. The overall method integrates ab initio electronic structure calculations with importance sampling techniques that permit the critical regions of configuration space to be determined. The computed ab initio energies and gradients are then accurately interpolated using neural networks (NN) rather than arbitrary parametrized analytical functional forms, moving interpolation or least-squares methods. The sampling method involves a tight integration of molecular dynamics calculations with neural networks that employ early stopping and regularization procedures to improve network performance and test for convergence. The procedure can be initiated using an empirical potential surface or direct dynamics. The accuracy and interpolation power of the method has been tested for two cases, the global potential surface for vinyl bromide undergoing unimolecular decomposition via four different reaction channels and nanometric cutting of silicon. The results show that the sampling methods permit the important regions of configuration space to be easily and rapidly identified, that convergence of the NN fit to the ab initio electronic structure database can be easily monitored, and that the interpolation accuracy of the NN fits is excellent, even for systems involving five atoms or more. The method permits a substantial computational speed and accuracy advantage over existing methods, is robust, and relatively easy to implement.

Parallel approach to identifying the well-test interpretation model using a neurocomputer

NASA Astrophysics Data System (ADS)

May, Edward A., Jr.; Dagli, Cihan H.

1996-03-01

The well test is one of the primary diagnostic and predictive tools used in the analysis of oil and gas wells. In these tests, a pressure recording device is placed in the well and the pressure response is recorded over time under controlled flow conditions. The interpreted results are indicators of the well's ability to flow and the damage done to the formation surrounding the wellbore during drilling and completion. The results are used for many purposes, including reservoir modeling (simulation) and economic forecasting. The first step in the analysis is the identification of the Well-Test Interpretation (WTI) model, which determines the appropriate solution method. Mis-identification of the WTI model occurs due to noise and non-ideal reservoir conditions. Previous studies have shown that a feed-forward neural network using the backpropagation algorithm can be used to identify the WTI model. One of the drawbacks to this approach is, however, training time, which can run into days of CPU time on personal computers. In this paper a similar neural network is applied using both a personal computer and a neurocomputer. Input data processing, network design, and performance are discussed and compared. The results show that the neurocomputer greatly eases the burden of training and allows the network to outperform a similar network running on a personal computer.

Cerebro-cerebellar interactions underlying temporal information processing.

PubMed

Aso, Kenji; Hanakawa, Takashi; Aso, Toshihiko; Fukuyama, Hidenao

2010-12-01

The neural basis of temporal information processing remains unclear, but it is proposed that the cerebellum plays an important role through its internal clock or feed-forward computation functions. In this study, fMRI was used to investigate the brain networks engaged in perceptual and motor aspects of subsecond temporal processing without accompanying coprocessing of spatial information. Direct comparison between perceptual and motor aspects of time processing was made with a categorical-design analysis. The right lateral cerebellum (lobule VI) was active during a time discrimination task, whereas the left cerebellar lobule VI was activated during a timed movement generation task. These findings were consistent with the idea that the cerebellum contributed to subsecond time processing in both perceptual and motor aspects. The feed-forward computational theory of the cerebellum predicted increased cerebro-cerebellar interactions during time information processing. In fact, a psychophysiological interaction analysis identified the supplementary motor and dorsal premotor areas, which had a significant functional connectivity with the right cerebellar region during a time discrimination task and with the left lateral cerebellum during a timed movement generation task. The involvement of cerebro-cerebellar interactions may provide supportive evidence that temporal information processing relies on the simulation of timing information through feed-forward computation in the cerebellum.

Reaction-diffusion-like formalism for plastic neural networks reveals dissipative solitons at criticality

NASA Astrophysics Data System (ADS)

Grytskyy, Dmytro; Diesmann, Markus; Helias, Moritz

2016-06-01

Self-organized structures in networks with spike-timing dependent synaptic plasticity (STDP) are likely to play a central role for information processing in the brain. In the present study we derive a reaction-diffusion-like formalism for plastic feed-forward networks of nonlinear rate-based model neurons with a correlation sensitive learning rule inspired by and being qualitatively similar to STDP. After obtaining equations that describe the change of the spatial shape of the signal from layer to layer, we derive a criterion for the nonlinearity necessary to obtain stable dynamics for arbitrary input. We classify the possible scenarios of signal evolution and find that close to the transition to the unstable regime metastable solutions appear. The form of these dissipative solitons is determined analytically and the evolution and interaction of several such coexistent objects is investigated.

Practical training framework for fitting a function and its derivatives.

PubMed

Pukrittayakamee, Arjpolson; Hagan, Martin; Raff, Lionel; Bukkapatnam, Satish T S; Komanduri, Ranga

2011-06-01

This paper describes a practical framework for using multilayer feedforward neural networks to simultaneously fit both a function and its first derivatives. This framework involves two steps. The first step is to train the network to optimize a performance index, which includes both the error in fitting the function and the error in fitting the derivatives. The second step is to prune the network by removing neurons that cause overfitting and then to retrain it. This paper describes two novel types of overfitting that are only observed when simultaneously fitting both a function and its first derivatives. A new pruning algorithm is proposed to eliminate these types of overfitting. Experimental results show that the pruning algorithm successfully eliminates the overfitting and produces the smoothest responses and the best generalization among all the training algorithms that we have tested.

Turbidity forecasting at a karst spring using combined machine learning and wavelet multiresolution analysis.

NASA Astrophysics Data System (ADS)

Savary, M.; Massei, N.; Johannet, A.; Dupont, J. P.; Hauchard, E.

2016-12-01

25% of the world populations drink water extracted from karst aquifer. The comprehension and the protection of these aquifers appear as crucial due to an increase of drinking water needs. In Normandie(North-West of France), the principal exploited aquifer is the chalk aquifer. The chalk aquifer highly karstified is an important water resource, regionally speaking. Connections between surface and underground waters thanks to karstification imply turbidity that decreases water quality. Both numerous parameters and phenomenons, and the non-linearity of the rainfall/turbidity relation influence the turbidity causing difficulties to model and forecast turbidity peaks. In this context, the Yport pumping well provides half of Le Havreconurbation drinking water supply (236 000 inhabitants). The aim of this work is thus to perform prediction of the turbidity peaks in order to help pumping well managers to decrease the impact of turbidity on water treatment. Database consists in hourly rainfalls coming from six rain gauges located on the alimentation basin since 2009 and hourly turbidity since 1993. Because of the lack of accurate physical description of the karst system and its surface basin, the systemic paradigm is chosen and a black box model: a neural network model is chosen. In a first step, correlation analyses are used to design the original model architecture by identifying the relation between output and input. The following optimization phases bring us four different architectures. These models were experimented to forecast 12h ahead turbidity and threshold surpassing. The first model is a simple multilayer perceptron. The second is a two-branches model designed to better represent the fast (rainfall) and low (evapotranspiration) dynamics. Each kind of model is developed using both a recurrent and feed-forward architecture. This work highlights that feed-forward multilayer perceptron is better to predict turbidity peaks when feed-forward two-branches model is better to predict threshold surpassing. In a second step, the implementation of wavelet decomposition within the neural network model to better apprehend slow and fast dynamics is tested and discussed, which could also allows accounting for non-linearity of the turbid response to some extent. This second approach is still under realization so far.

Non-linear Post Processing Image Enhancement

NASA Technical Reports Server (NTRS)

Hunt, Shawn; Lopez, Alex; Torres, Angel

1997-01-01

A non-linear filter for image post processing based on the feedforward Neural Network topology is presented. This study was undertaken to investigate the usefulness of "smart" filters in image post processing. The filter has shown to be useful in recovering high frequencies, such as those lost during the JPEG compression-decompression process. The filtered images have a higher signal to noise ratio, and a higher perceived image quality. Simulation studies comparing the proposed filter with the optimum mean square non-linear filter, showing examples of the high frequency recovery, and the statistical properties of the filter are given,

A neural network based artificial vision system for licence plate recognition.

PubMed

Draghici, S

1997-02-01

This paper presents a neural network based artificial vision system able to analyze the image of a car given by a camera, locate the registration plate and recognize the registration number of the car. The paper describes in detail various practical problems encountered in implementing this particular application and the solutions used to solve them. The main features of the system presented are: controlled stability-plasticity behavior, controlled reliability threshold, both off-line and on-line learning, self assessment of the output reliability and high reliability based on high level multiple feedback. The system has been designed using a modular approach. Sub-modules can be upgraded and/or substituted independently, thus making the system potentially suitable in a large variety of vision applications. The OCR engine was designed as an interchangeable plug-in module. This allows the user to choose an OCR engine which is suited to the particular application and to upgrade it easily in the future. At present, there are several versions of this OCR engine. One of them is based on a fully connected feedforward artificial neural network with sigmoidal activation functions. This network can be trained with various training algorithms such as error backpropagation. An alternative OCR engine is based on the constraint based decomposition (CBD) training architecture. The system has showed the following performances (on average) on real-world data: successful plate location and segmentation about 99%, successful character recognition about 98% and successful recognition of complete registration plates about 80%.

Artificial neural network assisted kinetic spectrophotometric technique for simultaneous determination of paracetamol and p-aminophenol in pharmaceutical samples using localized surface plasmon resonance band of silver nanoparticles

NASA Astrophysics Data System (ADS)

Khodaveisi, Javad; Dadfarnia, Shayessteh; Haji Shabani, Ali Mohammad; Rohani Moghadam, Masoud; Hormozi-Nezhad, Mohammad Reza

2015-03-01

Spectrophotometric analysis method based on the combination of the principal component analysis (PCA) with the feed-forward neural network (FFNN) and the radial basis function network (RBFN) was proposed for the simultaneous determination of paracetamol (PAC) and p-aminophenol (PAP). This technique relies on the difference between the kinetic rates of the reactions between analytes and silver nitrate as the oxidizing agent in the presence of polyvinylpyrrolidone (PVP) which is the stabilizer. The reactions are monitored at the analytical wavelength of 420 nm of the localized surface plasmon resonance (LSPR) band of the formed silver nanoparticles (Ag-NPs). Under the optimized conditions, the linear calibration graphs were obtained in the concentration range of 0.122-2.425 μg mL-1 for PAC and 0.021-5.245 μg mL-1 for PAP. The limit of detection in terms of standard approach (LODSA) and upper limit approach (LODULA) were calculated to be 0.027 and 0.032 μg mL-1 for PAC and 0.006 and 0.009 μg mL-1 for PAP. The important parameters were optimized for the artificial neural network (ANN) models. Statistical parameters indicated that the ability of the both methods is comparable. The proposed method was successfully applied to the simultaneous determination of PAC and PAP in pharmaceutical preparations.

Emergent spatial synaptic structure from diffusive plasticity.

PubMed

Sweeney, Yann; Clopath, Claudia

2017-04-01

Some neurotransmitters can diffuse freely across cell membranes, influencing neighbouring neurons regardless of their synaptic coupling. This provides a means of neural communication, alternative to synaptic transmission, which can influence the way in which neural networks process information. Here, we ask whether diffusive neurotransmission can also influence the structure of synaptic connectivity in a network undergoing plasticity. We propose a form of Hebbian synaptic plasticity which is mediated by a diffusive neurotransmitter. Whenever a synapse is modified at an individual neuron through our proposed mechanism, similar but smaller modifications occur in synapses connecting to neighbouring neurons. The effects of this diffusive plasticity are explored in networks of rate-based neurons. This leads to the emergence of spatial structure in the synaptic connectivity of the network. We show that this spatial structure can coexist with other forms of structure in the synaptic connectivity, such as with groups of strongly interconnected neurons that form in response to correlated external drive. Finally, we explore diffusive plasticity in a simple feedforward network model of receptive field development. We show that, as widely observed across sensory cortex, the preferred stimulus identity of neurons in our network become spatially correlated due to diffusion. Our proposed mechanism of diffusive plasticity provides an efficient mechanism for generating these spatial correlations in stimulus preference which can flexibly interact with other forms of synaptic organisation. © 2016 Federation of European Neuroscience Societies and John Wiley & Sons Ltd.

A novel artificial neural network method for biomedical prediction based on matrix pseudo-inversion.

PubMed

Cai, Binghuang; Jiang, Xia

2014-04-01

Biomedical prediction based on clinical and genome-wide data has become increasingly important in disease diagnosis and classification. To solve the prediction problem in an effective manner for the improvement of clinical care, we develop a novel Artificial Neural Network (ANN) method based on Matrix Pseudo-Inversion (MPI) for use in biomedical applications. The MPI-ANN is constructed as a three-layer (i.e., input, hidden, and output layers) feed-forward neural network, and the weights connecting the hidden and output layers are directly determined based on MPI without a lengthy learning iteration. The LASSO (Least Absolute Shrinkage and Selection Operator) method is also presented for comparative purposes. Single Nucleotide Polymorphism (SNP) simulated data and real breast cancer data are employed to validate the performance of the MPI-ANN method via 5-fold cross validation. Experimental results demonstrate the efficacy of the developed MPI-ANN for disease classification and prediction, in view of the significantly superior accuracy (i.e., the rate of correct predictions), as compared with LASSO. The results based on the real breast cancer data also show that the MPI-ANN has better performance than other machine learning methods (including support vector machine (SVM), logistic regression (LR), and an iterative ANN). In addition, experiments demonstrate that our MPI-ANN could be used for bio-marker selection as well. Copyright © 2013 Elsevier Inc. All rights reserved.

Using statistical and artificial neural network models to forecast potentiometric levels at a deep well in South Texas

NASA Astrophysics Data System (ADS)

Uddameri, V.

2007-01-01

Reliable forecasts of monthly and quarterly fluctuations in groundwater levels are necessary for short- and medium-term planning and management of aquifers to ensure proper service of seasonal demands within a region. Development of physically based transient mathematical models at this time scale poses considerable challenges due to lack of suitable data and other uncertainties. Artificial neural networks (ANN) possess flexible mathematical structures and are capable of mapping highly nonlinear relationships. Feed-forward neural network models were constructed and trained using the back-percolation algorithm to forecast monthly and quarterly time-series water levels at a well that taps into the deeper Evangeline formation of the Gulf Coast aquifer in Victoria, TX. Unlike unconfined formations, no causal relationships exist between water levels and hydro-meteorological variables measured near the vicinity of the well. As such, an endogenous forecasting model using dummy variables to capture short-term seasonal fluctuations and longer-term (decadal) trends was constructed. The root mean square error, mean absolute deviation and correlation coefficient ( R) were noted to be 1.40, 0.33 and 0.77 m, respectively, for an evaluation dataset of quarterly measurements and 1.17, 0.46, and 0.88 m for an evaluative monthly dataset not used to train or test the model. These statistics were better for the ANN model than those developed using statistical regression techniques.

Feedback to distal dendrites links fMRI signals to neural receptive fields in a spiking network model of the visual cortex.

PubMed

Heikkinen, Hanna; Sharifian, Fariba; Vigario, Ricardo; Vanni, Simo

2015-07-01

The blood oxygenation level-dependent (BOLD) response has been strongly associated with neuronal activity in the brain. However, some neuronal tuning properties are consistently different from the BOLD response. We studied the spatial extent of neural and hemodynamic responses in the primary visual cortex, where the BOLD responses spread and interact over much longer distances than the small receptive fields of individual neurons would predict. Our model shows that a feedforward-feedback loop between V1 and a higher visual area can account for the observed spread of the BOLD response. In particular, anisotropic landing of inputs to compartmental neurons were necessary to account for the BOLD signal spread, while retaining realistic spiking responses. Our work shows that simple dendrites can separate tuning at the synapses and at the action potential output, thus bridging the BOLD signal to the neural receptive fields with high fidelity. Copyright © 2015 the American Physiological Society.

Classification of a Driver's cognitive workload levels using artificial neural network on ECG signals.

PubMed

Tjolleng, Amir; Jung, Kihyo; Hong, Wongi; Lee, Wonsup; Lee, Baekhee; You, Heecheon; Son, Joonwoo; Park, Seikwon

2017-03-01

An artificial neural network (ANN) model was developed in the present study to classify the level of a driver's cognitive workload based on electrocardiography (ECG). ECG signals were measured on 15 male participants while they performed a simulated driving task as a primary task with/without an N-back task as a secondary task. Three time-domain ECG measures (mean inter-beat interval (IBI), standard deviation of IBIs, and root mean squared difference of adjacent IBIs) and three frequencydomain ECG measures (power in low frequency, power in high frequency, and ratio of power in low and high frequencies) were calculated. To compensate for individual differences in heart response during the driving tasks, a three-step data processing procedure was performed to ECG signals of each participant: (1) selection of two most sensitive ECG measures, (2) definition of three (low, medium, and high) cognitive workload levels, and (3) normalization of the selected ECG measures. An ANN model was constructed using a feed-forward network and scaled conjugate gradient as a back-propagation learning rule. The accuracy of the ANN classification model was found satisfactory for learning data (95%) and testing data (82%). Copyright © 2016 Elsevier Ltd. All rights reserved.

Pattern recognition and classification of vibrational spectra by artificial neural networks

NASA Astrophysics Data System (ADS)

Yang, Husheng

1999-10-01

A drawback of current open-path Fourier transform infrared (OP/FT-IR) systems is that they need a human expert to determine those compounds that may be quantified from a given spectrum. In this study, three types of artificial neural networks were used to alleviate this problem. Firstly, multi-layer feed-forward neural networks were used to automatically recognize compounds in an OP/FT-IR spectrum. Each neural network was trained to recognize one compound in the presence of up to ten interferents in an OP/FT-IR spectrum. The networks were successfully used to recognize five alcohols and two chlorinated compounds in field-measured controlled-release OP/FT-IR spectra of mixtures of these compounds. It has also been demonstrated that a neural network could correctly identify a spectrum in the presence of an interferent that was not included in the training set and could also reject interferents it has not seen before. Secondly, the possibility of using one- and two- dimensional Kohonen self-organizing maps (SOMs) to recognize similarities in low-resolution vapor-phase infrared spectra without any additional information has been investigated. Both full-range reference spectra and open-path window reference spectra were used to train the networks and the trained networks were then used to classify the reference spectra into several groups. The results showed that the SOMs obtained from the two different training sets were quite different, and it is more appropriate to use the second SOM in OP/FT-IR spectrometry. Thirdly, vapor-phase FT-IR reference spectra of five alcohols along with four baseline spectra were encoded as prototype vectors for a Hopfield network. Inclusion of the baseline spectra allowed the network to classify spectra as unknowns, when the reference spectra of these compounds were not stored as prototype vectors in the network. The network could identify each of the 5 alcohols correctly even in the presence of noise and interfering compounds. Finally, one- and two-dimensional Kohonen SOMs were also successfully used for the unsupervised differentiation of the Fourier transform Raman spectra of hardwoods from softwoods. A semi-quantitative method that is based on the Euclidean distances of the weight matrix has been developed to assist the automatic clustering of the neurons in a two-dimensional SOM.

Brain-machine interface control of a manipulator using small-world neural network and shared control strategy.

PubMed

Li, Ting; Hong, Jun; Zhang, Jinhua; Guo, Feng

2014-03-15

The improvement of the resolution of brain signal and the ability to control external device has been the most important goal in BMI research field. This paper describes a non-invasive brain-actuated manipulator experiment, which defined a paradigm for the motion control of a serial manipulator based on motor imagery and shared control. The techniques of component selection, spatial filtering and classification of motor imagery were involved. Small-world neural network (SWNN) was used to classify five brain states. To verify the effectiveness of the proposed classifier, we replace the SWNN classifier by a radial basis function (RBF) networks neural network, a standard multi-layered feed-forward backpropagation network (SMN) and a multi-SVM classifier, with the same features for the classification. The results also indicate that the proposed classifier achieves a 3.83% improvement over the best results of other classifiers. We proposed a shared control method consisting of two control patterns to expand the control of BMI from the software angle. The job of path building for reaching the 'end' point was designated as an assessment task. We recorded all paths contributed by subjects and picked up relevant parameters as evaluation coefficients. With the assistance of two control patterns and series of machine learning algorithms, the proposed BMI originally achieved the motion control of a manipulator in the whole workspace. According to experimental results, we confirmed the feasibility of the proposed BMI method for 3D motion control of a manipulator using EEG during motor imagery. Copyright © 2013 Elsevier B.V. All rights reserved.

Prediction of Austenite Formation Temperatures Using Artificial Neural Networks

NASA Astrophysics Data System (ADS)

Schulze, P.; Schmidl, E.; Grund, T.; Lampke, T.

2016-03-01

For the modeling and design of heat treatments, in consideration of the development/ transformation of the microstructure, different material data depending on the chemical composition, the respective microstructure/phases and the temperature are necessary. Material data are, e.g. the thermal conductivity, heat capacity, thermal expansion and transformation data etc. The quality of thermal simulations strongly depends on the accuracy of the material data. For many materials, the required data - in particular for different microstructures and temperatures - are rare in the literature. In addition, a different chemical composition within the permitted limits of the considered steel alloy cannot be predicted. A solution for this problem is provided by the calculation of material data using Artificial Neural Networks (ANN). In the present study, the start and finish temperatures of the transformation from the bcc lattice to the fcc lattice structure of hypoeutectoid steels are calculated using an Artificial Neural Network. An appropriate database containing different transformation temperatures (austenite formation temperatures) to train the ANN is selected from the literature. In order to find a suitable feedforward network, the network topologies as well as the activation functions of the hidden layers are varied and subsequently evaluated in terms of the prediction accuracy. The transformation temperatures calculated by the ANN exhibit a very good compliance compared to the experimental data. The results show that the prediction performance is even higher compared to classical empirical equations such as Andrews or Brandis. Therefore, it can be assumed that the presented ANN is a convenient tool to distinguish between bcc and fcc phases in hypoeutectoid steels.

Decorrelation of Neural-Network Activity by Inhibitory Feedback

PubMed Central

Einevoll, Gaute T.; Diesmann, Markus

2012-01-01

Correlations in spike-train ensembles can seriously impair the encoding of information by their spatio-temporal structure. An inevitable source of correlation in finite neural networks is common presynaptic input to pairs of neurons. Recent studies demonstrate that spike correlations in recurrent neural networks are considerably smaller than expected based on the amount of shared presynaptic input. Here, we explain this observation by means of a linear network model and simulations of networks of leaky integrate-and-fire neurons. We show that inhibitory feedback efficiently suppresses pairwise correlations and, hence, population-rate fluctuations, thereby assigning inhibitory neurons the new role of active decorrelation. We quantify this decorrelation by comparing the responses of the intact recurrent network (feedback system) and systems where the statistics of the feedback channel is perturbed (feedforward system). Manipulations of the feedback statistics can lead to a significant increase in the power and coherence of the population response. In particular, neglecting correlations within the ensemble of feedback channels or between the external stimulus and the feedback amplifies population-rate fluctuations by orders of magnitude. The fluctuation suppression in homogeneous inhibitory networks is explained by a negative feedback loop in the one-dimensional dynamics of the compound activity. Similarly, a change of coordinates exposes an effective negative feedback loop in the compound dynamics of stable excitatory-inhibitory networks. The suppression of input correlations in finite networks is explained by the population averaged correlations in the linear network model: In purely inhibitory networks, shared-input correlations are canceled by negative spike-train correlations. In excitatory-inhibitory networks, spike-train correlations are typically positive. Here, the suppression of input correlations is not a result of the mere existence of correlations between excitatory (E) and inhibitory (I) neurons, but a consequence of a particular structure of correlations among the three possible pairings (EE, EI, II). PMID:23133368

An Efficient Supervised Training Algorithm for Multilayer Spiking Neural Networks

PubMed Central

Xie, Xiurui; Qu, Hong; Liu, Guisong; Zhang, Malu; Kurths, Jürgen

2016-01-01

The spiking neural networks (SNNs) are the third generation of neural networks and perform remarkably well in cognitive tasks such as pattern recognition. The spike emitting and information processing mechanisms found in biological cognitive systems motivate the application of the hierarchical structure and temporal encoding mechanism in spiking neural networks, which have exhibited strong computational capability. However, the hierarchical structure and temporal encoding approach require neurons to process information serially in space and time respectively, which reduce the training efficiency significantly. For training the hierarchical SNNs, most existing methods are based on the traditional back-propagation algorithm, inheriting its drawbacks of the gradient diffusion and the sensitivity on parameters. To keep the powerful computation capability of the hierarchical structure and temporal encoding mechanism, but to overcome the low efficiency of the existing algorithms, a new training algorithm, the Normalized Spiking Error Back Propagation (NSEBP) is proposed in this paper. In the feedforward calculation, the output spike times are calculated by solving the quadratic function in the spike response model instead of detecting postsynaptic voltage states at all time points in traditional algorithms. Besides, in the feedback weight modification, the computational error is propagated to previous layers by the presynaptic spike jitter instead of the gradient decent rule, which realizes the layer-wised training. Furthermore, our algorithm investigates the mathematical relation between the weight variation and voltage error change, which makes the normalization in the weight modification applicable. Adopting these strategies, our algorithm outperforms the traditional SNN multi-layer algorithms in terms of learning efficiency and parameter sensitivity, that are also demonstrated by the comprehensive experimental results in this paper. PMID:27044001

Deep neural networks show an equivalent and often superior performance to dermatologists in onychomycosis diagnosis: Automatic construction of onychomycosis datasets by region-based convolutional deep neural network.

PubMed

Han, Seung Seog; Park, Gyeong Hun; Lim, Woohyung; Kim, Myoung Shin; Na, Jung Im; Park, Ilwoo; Chang, Sung Eun

2018-01-01

Although there have been reports of the successful diagnosis of skin disorders using deep learning, unrealistically large clinical image datasets are required for artificial intelligence (AI) training. We created datasets of standardized nail images using a region-based convolutional neural network (R-CNN) trained to distinguish the nail from the background. We used R-CNN to generate training datasets of 49,567 images, which we then used to fine-tune the ResNet-152 and VGG-19 models. The validation datasets comprised 100 and 194 images from Inje University (B1 and B2 datasets, respectively), 125 images from Hallym University (C dataset), and 939 images from Seoul National University (D dataset). The AI (ensemble model; ResNet-152 + VGG-19 + feedforward neural networks) results showed test sensitivity/specificity/ area under the curve values of (96.0 / 94.7 / 0.98), (82.7 / 96.7 / 0.95), (92.3 / 79.3 / 0.93), (87.7 / 69.3 / 0.82) for the B1, B2, C, and D datasets. With a combination of the B1 and C datasets, the AI Youden index was significantly (p = 0.01) higher than that of 42 dermatologists doing the same assessment manually. For B1+C and B2+ D dataset combinations, almost none of the dermatologists performed as well as the AI. By training with a dataset comprising 49,567 images, we achieved a diagnostic accuracy for onychomycosis using deep learning that was superior to that of most of the dermatologists who participated in this study.

A biologically inspired neural net for trajectory formation and obstacle avoidance.

PubMed

Glasius, R; Komoda, A; Gielen, S C

1996-06-01

In this paper we present a biologically inspired two-layered neural network for trajectory formation and obstacle avoidance. The two topographically ordered neural maps consist of analog neurons having continuous dynamics. The first layer, the sensory map, receives sensory information and builds up an activity pattern which contains the optimal solution (i.e. shortest path without collisions) for any given set of current position, target positions and obstacle positions. Targets and obstacles are allowed to move, in which case the activity pattern in the sensory map will change accordingly. The time evolution of the neural activity in the second layer, the motor map, results in a moving cluster of activity, which can be interpreted as a population vector. Through the feedforward connections between the two layers, input of the sensory map directs the movement of the cluster along the optimal path from the current position of the cluster to the target position. The smooth trajectory is the result of the intrinsic dynamics of the network only. No supervisor is required. The output of the motor map can be used for direct control of an autonomous system in a cluttered environment or for control of the actuators of a biological limb or robot manipulator. The system is able to reach a target even in the presence of an external perturbation. Computer simulations of a point robot and a multi-joint manipulator illustrate the theory.

Automated detection of videotaped neonatal seizures of epileptic origin.

PubMed

Karayiannis, Nicolaos B; Xiong, Yaohua; Tao, Guozhi; Frost, James D; Wise, Merrill S; Hrachovy, Richard A; Mizrahi, Eli M

2006-06-01

This study aimed at the development of a seizure-detection system by training neural networks with quantitative motion information extracted from short video segments of neonatal seizures of the myoclonic and focal clonic types and random infant movements. The motion of the infants' body parts was quantified by temporal motion-strength signals extracted from video segments by motion-segmentation methods based on optical flow computation. The area of each frame occupied by the infants' moving body parts was segmented by clustering the motion parameters obtained by fitting an affine model to the pixel velocities. The motion of the infants' body parts also was quantified by temporal motion-trajectory signals extracted from video recordings by robust motion trackers based on block-motion models. These motion trackers were developed to adjust autonomously to illumination and contrast changes that may occur during the video-frame sequence. Video segments were represented by quantitative features obtained by analyzing motion-strength and motion-trajectory signals in both the time and frequency domains. Seizure recognition was performed by conventional feed-forward neural networks, quantum neural networks, and cosine radial basis function neural networks, which were trained to detect neonatal seizures of the myoclonic and focal clonic types and to distinguish them from random infant movements. The computational tools and procedures developed for automated seizure detection were evaluated on a set of 240 video segments of 54 patients exhibiting myoclonic seizures (80 segments), focal clonic seizures (80 segments), and random infant movements (80 segments). Regardless of the decision scheme used for interpreting the responses of the trained neural networks, all the neural network models exhibited sensitivity and specificity>90%. For one of the decision schemes proposed for interpreting the responses of the trained neural networks, the majority of the trained neural-network models exhibited sensitivity>90% and specificity>95%. In particular, cosine radial basis function neural networks achieved the performance targets of this phase of the project (i.e., sensitivity>95% and specificity>95%). The best among the motion segmentation and tracking methods developed in this study produced quantitative features that constitute a reliable basis for detecting neonatal seizures. The performance targets of this phase of the project were achieved by combining the quantitative features obtained by analyzing motion-strength signals with those produced by analyzing motion-trajectory signals. The computational procedures and tools developed in this study to perform off-line analysis of short video segments will be used in the next phase of this project, which involves the integration of these procedures and tools into a system that can process and analyze long video recordings of infants monitored for seizures in real time.

Extracting functionally feedforward networks from a population of spiking neurons

PubMed Central

Vincent, Kathleen; Tauskela, Joseph S.; Thivierge, Jean-Philippe

2012-01-01

Neuronal avalanches are a ubiquitous form of activity characterized by spontaneous bursts whose size distribution follows a power-law. Recent theoretical models have replicated power-law avalanches by assuming the presence of functionally feedforward connections (FFCs) in the underlying dynamics of the system. Accordingly, avalanches are generated by a feedforward chain of activation that persists despite being embedded in a larger, massively recurrent circuit. However, it is unclear to what extent networks of living neurons that exhibit power-law avalanches rely on FFCs. Here, we employed a computational approach to reconstruct the functional connectivity of cultured cortical neurons plated on multielectrode arrays (MEAs) and investigated whether pharmacologically induced alterations in avalanche dynamics are accompanied by changes in FFCs. This approach begins by extracting a functional network of directed links between pairs of neurons, and then evaluates the strength of FFCs using Schur decomposition. In a first step, we examined the ability of this approach to extract FFCs from simulated spiking neurons. The strength of FFCs obtained in strictly feedforward networks diminished monotonically as links were gradually rewired at random. Next, we estimated the FFCs of spontaneously active cortical neuron cultures in the presence of either a control medium, a GABAA receptor antagonist (PTX), or an AMPA receptor antagonist combined with an NMDA receptor antagonist (APV/DNQX). The distribution of avalanche sizes in these cultures was modulated by this pharmacology, with a shallower power-law under PTX (due to the prominence of larger avalanches) and a steeper power-law under APV/DNQX (due to avalanches recruiting fewer neurons) relative to control cultures. The strength of FFCs increased in networks after application of PTX, consistent with an amplification of feedforward activity during avalanches. Conversely, FFCs decreased after application of APV/DNQX, consistent with fading feedforward activation. The observed alterations in FFCs provide experimental support for recent theoretical work linking power-law avalanches to the feedforward organization of functional connections in local neuronal circuits. PMID:23091458

Extracting functionally feedforward networks from a population of spiking neurons.

PubMed

Vincent, Kathleen; Tauskela, Joseph S; Thivierge, Jean-Philippe

2012-01-01

Neuronal avalanches are a ubiquitous form of activity characterized by spontaneous bursts whose size distribution follows a power-law. Recent theoretical models have replicated power-law avalanches by assuming the presence of functionally feedforward connections (FFCs) in the underlying dynamics of the system. Accordingly, avalanches are generated by a feedforward chain of activation that persists despite being embedded in a larger, massively recurrent circuit. However, it is unclear to what extent networks of living neurons that exhibit power-law avalanches rely on FFCs. Here, we employed a computational approach to reconstruct the functional connectivity of cultured cortical neurons plated on multielectrode arrays (MEAs) and investigated whether pharmacologically induced alterations in avalanche dynamics are accompanied by changes in FFCs. This approach begins by extracting a functional network of directed links between pairs of neurons, and then evaluates the strength of FFCs using Schur decomposition. In a first step, we examined the ability of this approach to extract FFCs from simulated spiking neurons. The strength of FFCs obtained in strictly feedforward networks diminished monotonically as links were gradually rewired at random. Next, we estimated the FFCs of spontaneously active cortical neuron cultures in the presence of either a control medium, a GABA(A) receptor antagonist (PTX), or an AMPA receptor antagonist combined with an NMDA receptor antagonist (APV/DNQX). The distribution of avalanche sizes in these cultures was modulated by this pharmacology, with a shallower power-law under PTX (due to the prominence of larger avalanches) and a steeper power-law under APV/DNQX (due to avalanches recruiting fewer neurons) relative to control cultures. The strength of FFCs increased in networks after application of PTX, consistent with an amplification of feedforward activity during avalanches. Conversely, FFCs decreased after application of APV/DNQX, consistent with fading feedforward activation. The observed alterations in FFCs provide experimental support for recent theoretical work linking power-law avalanches to the feedforward organization of functional connections in local neuronal circuits.

Incorporating geographical factors with artificial neural networks to predict reference values of erythrocyte sedimentation rate

PubMed Central

2013-01-01

Background The measurement of the Erythrocyte Sedimentation Rate (ESR) value is a standard procedure performed during a typical blood test. In order to formulate a unified standard of establishing reference ESR values, this paper presents a novel prediction model in which local normal ESR values and corresponding geographical factors are used to predict reference ESR values using multi-layer feed-forward artificial neural networks (ANN). Methods and findings Local normal ESR values were obtained from hospital data, while geographical factors that include altitude, sunshine hours, relative humidity, temperature and precipitation were obtained from the National Geographical Data Information Centre in China. The results show that predicted values are statistically in agreement with measured values. Model results exhibit significant agreement between training data and test data. Consequently, the model is used to predict the unseen local reference ESR values. Conclusions Reference ESR values can be established with geographical factors by using artificial intelligence techniques. ANN is an effective method for simulating and predicting reference ESR values because of its ability to model nonlinear and complex relationships. PMID:23497145

Neural-network hybrid control for antilock braking systems.

PubMed

Lin, Chih-Min; Hsu, C F

2003-01-01

The antilock braking systems are designed to maximize wheel traction by preventing the wheels from locking during braking, while also maintaining adequate vehicle steerability; however, the performance is often degraded under harsh road conditions. In this paper, a hybrid control system with a recurrent neural network (RNN) observer is developed for antilock braking systems. This hybrid control system is comprised of an ideal controller and a compensation controller. The ideal controller, containing an RNN uncertainty observer, is the principal controller; and the compensation controller is a compensator for the difference between the system uncertainty and the estimated uncertainty. Since for dynamic response the RNN has capabilities superior to the feedforward NN, it is utilized for the uncertainty observer. The Taylor linearization technique is employed to increase the learning ability of the RNN. In addition, the on-line parameter adaptation laws are derived based on a Lyapunov function, so the stability of the system can be guaranteed. Simulations are performed to demonstrate the effectiveness of the proposed NN hybrid control system for antilock braking control under various road conditions.

Incorporating geographical factors with artificial neural networks to predict reference values of erythrocyte sedimentation rate.

PubMed

Yang, Qingsheng; Mwenda, Kevin M; Ge, Miao

2013-03-12

The measurement of the Erythrocyte Sedimentation Rate (ESR) value is a standard procedure performed during a typical blood test. In order to formulate a unified standard of establishing reference ESR values, this paper presents a novel prediction model in which local normal ESR values and corresponding geographical factors are used to predict reference ESR values using multi-layer feed-forward artificial neural networks (ANN). Local normal ESR values were obtained from hospital data, while geographical factors that include altitude, sunshine hours, relative humidity, temperature and precipitation were obtained from the National Geographical Data Information Centre in China.The results show that predicted values are statistically in agreement with measured values. Model results exhibit significant agreement between training data and test data. Consequently, the model is used to predict the unseen local reference ESR values. Reference ESR values can be established with geographical factors by using artificial intelligence techniques. ANN is an effective method for simulating and predicting reference ESR values because of its ability to model nonlinear and complex relationships.

Learning the inverse kinetics of an octopus-like manipulator in three-dimensional space.

PubMed

Giorelli, M; Renda, F; Calisti, M; Arienti, A; Ferri, G; Laschi, C

2015-05-13

This work addresses the inverse kinematics problem of a bioinspired octopus-like manipulator moving in three-dimensional space. The bioinspired manipulator has a conical soft structure that confers the ability of twirling around objects as a real octopus arm does. Despite the simple design, the soft conical shape manipulator driven by cables is described by nonlinear differential equations, which are difficult to solve analytically. Since exact solutions of the equations are not available, the Jacobian matrix cannot be calculated analytically and the classical iterative methods cannot be used. To overcome the intrinsic problems of methods based on the Jacobian matrix, this paper proposes a neural network learning the inverse kinematics of a soft octopus-like manipulator driven by cables. After the learning phase, a feed-forward neural network is able to represent the relation between manipulator tip positions and forces applied to the cables. Experimental results show that a desired tip position can be achieved in a short time, since heavy computations are avoided, with a degree of accuracy of 8% relative average error with respect to the total arm length.

Coding the presence of visual objects in a recurrent neural network of visual cortex.

PubMed

Zwickel, Timm; Wachtler, Thomas; Eckhorn, Reinhard

2007-01-01

Before we can recognize a visual object, our visual system has to segregate it from its background. This requires a fast mechanism for establishing the presence and location of objects independently of their identity. Recently, border-ownership neurons were recorded in monkey visual cortex which might be involved in this task [Zhou, H., Friedmann, H., von der Heydt, R., 2000. Coding of border ownership in monkey visual cortex. J. Neurosci. 20 (17), 6594-6611]. In order to explain the basic mechanisms required for fast coding of object presence, we have developed a neural network model of visual cortex consisting of three stages. Feed-forward and lateral connections support coding of Gestalt properties, including similarity, good continuation, and convexity. Neurons of the highest area respond to the presence of an object and encode its position, invariant of its form. Feedback connections to the lowest area facilitate orientation detectors activated by contours belonging to potential objects, and thus generate the experimentally observed border-ownership property. This feedback control acts fast and significantly improves the figure-ground segregation required for the consecutive task of object recognition.

Estimation of biogas and methane yields in an UASB treating potato starch processing wastewater with backpropagation artificial neural network.

PubMed

Antwi, Philip; Li, Jianzheng; Boadi, Portia Opoku; Meng, Jia; Shi, En; Deng, Kaiwen; Bondinuba, Francis Kwesi

2017-03-01

Three-layered feedforward backpropagation (BP) artificial neural networks (ANN) and multiple nonlinear regression (MnLR) models were developed to estimate biogas and methane yield in an upflow anaerobic sludge blanket (UASB) reactor treating potato starch processing wastewater (PSPW). Anaerobic process parameters were optimized to identify their importance on methanation. pH, total chemical oxygen demand, ammonium, alkalinity, total Kjeldahl nitrogen, total phosphorus, volatile fatty acids and hydraulic retention time selected based on principal component analysis were used as input variables, whiles biogas and methane yield were employed as target variables. Quasi-Newton method and conjugate gradient backpropagation algorithms were best among eleven training algorithms. Coefficient of determination (R 2 ) of the BP-ANN reached 98.72% and 97.93% whiles MnLR model attained 93.9% and 91.08% for biogas and methane yield, respectively. Compared with the MnLR model, BP-ANN model demonstrated significant performance, suggesting possible control of the anaerobic digestion process with the BP-ANN model. Copyright © 2016 Elsevier Ltd. All rights reserved.

Monthly monsoon rainfall forecasting using artificial neural networks

NASA Astrophysics Data System (ADS)

Ganti, Ravikumar

2014-10-01

Indian agriculture sector heavily depends on monsoon rainfall for successful harvesting. In the past, prediction of rainfall was mainly performed using regression models, which provide reasonable accuracy in the modelling and forecasting of complex physical systems. Recently, Artificial Neural Networks (ANNs) have been proposed as efficient tools for modelling and forecasting. A feed-forward multi-layer perceptron type of ANN architecture trained using the popular back-propagation algorithm was employed in this study. Other techniques investigated for modeling monthly monsoon rainfall include linear and non-linear regression models for comparison purposes. The data employed in this study include monthly rainfall and monthly average of the daily maximum temperature in the North Central region in India. Specifically, four regression models and two ANN model's were developed. The performance of various models was evaluated using a wide variety of standard statistical parameters and scatter plots. The results obtained in this study for forecasting monsoon rainfalls using ANNs have been encouraging. India's economy and agricultural activities can be effectively managed with the help of the availability of the accurate monsoon rainfall forecasts.

Deforestation and Industrial Forest Patterns in Colombia: a Case Study

NASA Astrophysics Data System (ADS)

Huo, L. Z.; Boschetti, L.; Sparks, A. M.; Clerici, N.

2017-12-01

The recent peace agreement between the government and the Revolutionary Armed Forces of Colombia (FARC) offers new opportunities for peaceful and sustainable development, but at the same time requires a timely effort to protect biological resources, and ecosystem services (Clerici et al., 2016). In this context, we use the 2001-2017 Landsat data record to prototype a methodology to establish a baseline of deforestation, afforestation and industrial forest practices (i.e. establishment and harvest of forest plantations), and to monitor future changes. Two study areas, which have seen considerable deforestation in recent years, were selected: one in the South of the country, at the edge of the Amazon Forest (WRS path 008 row 059) and one in the center, in mixed forest (WRS path 008 row 055). The time series of all the available cloud free Landsat 5, Landsat 7 and Landsat 8 data was classified into a sequence of binary forest/non forest maps using a deep learning model, successfully used in the natural language processing field, trained to detect forest transitions. Recurrent Neural Networks (RNN) is a class of artificial neural network that extends the conventional neural network with loops in the connections (Graves et al., 2013). Unlike a feed-forward neural network, an RNN is able to process the sequential inputs by having a recurrent hidden state whose activation at each step depends on that of the previous steps. In this manner, the RNN provides a good framework to dynamically model time series data, and has been successfully applied to natural language processing in Google (Sutskever et al., 2014). The sequence of forest cover state maps was subsequently post-processed to differentiate between deforestation (e.g. transition from forest to non forest land use) and industrial forest harvest (i.e. timber harvest followed by regrowth), by integrating the detection of temporal patterns, and spatial patterns. References Clerici, N., et al., (2016). Colombia: Dealing in conservation. Science, 354(6309), 190-190. Sutskever I.,et al. (2014). Sequence to sequence learning with neural networks. Advances in neural information processing systems, 3104-3112. Graves A., et al. (2013). Speech recognition with deep recurrent neural networks. In Proc. IEEE Int. Conf. Acoust. Speech Signal Process., 6645-6649.

Ship localization in Santa Barbara Channel using machine learning classifiers.

PubMed

Niu, Haiqiang; Ozanich, Emma; Gerstoft, Peter

2017-11-01

Machine learning classifiers are shown to outperform conventional matched field processing for a deep water (600 m depth) ocean acoustic-based ship range estimation problem in the Santa Barbara Channel Experiment when limited environmental information is known. Recordings of three different ships of opportunity on a vertical array were used as training and test data for the feed-forward neural network and support vector machine classifiers, demonstrating the feasibility of machine learning methods to locate unseen sources. The classifiers perform well up to 10 km range whereas the conventional matched field processing fails at about 4 km range without accurate environmental information.

Application of DBNs for concerned internet information detecting

NASA Astrophysics Data System (ADS)

Wang, Yanfang; Gao, Song

2017-03-01

In recent years, deep learning has achieved great success in many fields, ranging from voice recognition and image classification to computer vision. In this study we apply DBNs to concerned internet information in Chinese detecting problem, since there are inherent differences between English and Chinese. Contrastive divergence (CD) is employed in the DBNs to learn a multi-layer generative model from numerous unlabeled data. The features obtained by this model are used to initialize the feed-forward neural network, which can be fine-tuned with backpropagation. Experiment results indicate that, the model and training method we proposed can be used to detect the concerned internet information effectively and accurately.

Simulation of a Multidimensional Input Quantum Perceptron

NASA Astrophysics Data System (ADS)

Yamamoto, Alexandre Y.; Sundqvist, Kyle M.; Li, Peng; Harris, H. Rusty

2018-06-01

In this work, we demonstrate the improved data separation capabilities of the Multidimensional Input Quantum Perceptron (MDIQP), a fundamental cell for the construction of more complex Quantum Artificial Neural Networks (QANNs). This is done by using input controlled alterations of ancillary qubits in combination with phase estimation and learning algorithms. The MDIQP is capable of processing quantum information and classifying multidimensional data that may not be linearly separable, extending the capabilities of the classical perceptron. With this powerful component, we get much closer to the achievement of a feedforward multilayer QANN, which would be able to represent and classify arbitrary sets of data (both quantum and classical).

Effect of filtration of signals of brain activity on quality of recognition of brain activity patterns using artificial intelligence methods

NASA Astrophysics Data System (ADS)

Hramov, Alexander E.; Frolov, Nikita S.; Musatov, Vyachaslav Yu.

2018-02-01

In present work we studied features of the human brain states classification, corresponding to the real movements of hands and legs. For this purpose we used supervised learning algorithm based on feed-forward artificial neural networks (ANNs) with error back-propagation along with the support vector machine (SVM) method. We compared the quality of operator movements classification by means of EEG signals obtained experimentally in the absence of preliminary processing and after filtration in different ranges up to 25 Hz. It was shown that low-frequency filtering of multichannel EEG data significantly improved accuracy of operator movements classification.

Generation of the Human Biped Stance by a Neural Controller Able to Compensate Neurological Time Delay

PubMed Central

Jiang, Ping; Chiba, Ryosuke; Takakusaki, Kaoru; Ota, Jun

2016-01-01

The development of a physiologically plausible computational model of a neural controller that can realize a human-like biped stance is important for a large number of potential applications, such as assisting device development and designing robotic control systems. In this paper, we develop a computational model of a neural controller that can maintain a musculoskeletal model in a standing position, while incorporating a 120-ms neurological time delay. Unlike previous studies that have used an inverted pendulum model, a musculoskeletal model with seven joints and 70 muscular-tendon actuators is adopted to represent the human anatomy. Our proposed neural controller is composed of both feed-forward and feedback controls. The feed-forward control corresponds to the constant activation input necessary for the musculoskeletal model to maintain a standing posture. This compensates for gravity and regulates stiffness. The developed neural controller model can replicate two salient features of the human biped stance: (1) physiologically plausible muscle activations for quiet standing; and (2) selection of a low active stiffness for low energy consumption. PMID:27655271

A mixed incoherent feed-forward loop contributes to the regulation of bacterial photosynthesis genes.

PubMed

Mank, Nils N; Berghoff, Bork A; Klug, Gabriele

2013-03-01

Living cells use a variety of regulatory network motifs for accurate gene expression in response to changes in their environment or during differentiation processes. In Rhodobacter sphaeroides, a complex regulatory network controls expression of photosynthesis genes to guarantee optimal energy supply on one hand and to avoid photooxidative stress on the other hand. Recently, we identified a mixed incoherent feed-forward loop comprising the transcription factor PrrA, the sRNA PcrZ and photosynthesis target genes as part of this regulatory network. This point-of-view provides a comparison to other described feed-forward loops and discusses the physiological relevance of PcrZ in more detail.

A mixed incoherent feed-forward loop contributes to the regulation of bacterial photosynthesis genes

PubMed Central

Mank, Nils N.; Berghoff, Bork A.; Klug, Gabriele

2013-01-01

Living cells use a variety of regulatory network motifs for accurate gene expression in response to changes in their environment or during differentiation processes. In Rhodobacter sphaeroides, a complex regulatory network controls expression of photosynthesis genes to guarantee optimal energy supply on one hand and to avoid photooxidative stress on the other hand. Recently, we identified a mixed incoherent feed-forward loop comprising the transcription factor PrrA, the sRNA PcrZ and photosynthesis target genes as part of this regulatory network. This point-of-view provides a comparison to other described feed-forward loops and discusses the physiological relevance of PcrZ in more detail. PMID:23392242

Direct process estimation from tomographic data using artificial neural systems

NASA Astrophysics Data System (ADS)

Mohamad-Saleh, Junita; Hoyle, Brian S.; Podd, Frank J.; Spink, D. M.

2001-07-01

The paper deals with the goal of component fraction estimation in multicomponent flows, a critical measurement in many processes. Electrical capacitance tomography (ECT) is a well-researched sensing technique for this task, due to its low-cost, non-intrusion, and fast response. However, typical systems, which include practicable real-time reconstruction algorithms, give inaccurate results, and existing approaches to direct component fraction measurement are flow-regime dependent. In the investigation described, an artificial neural network approach is used to directly estimate the component fractions in gas-oil, gas-water, and gas-oil-water flows from ECT measurements. A 2D finite- element electric field model of a 12-electrode ECT sensor is used to simulate ECT measurements of various flow conditions. The raw measurements are reduced to a mutually independent set using principal components analysis and used with their corresponding component fractions to train multilayer feed-forward neural networks (MLFFNNs). The trained MLFFNNs are tested with patterns consisting of unlearned ECT simulated and plant measurements. Results included in the paper have a mean absolute error of less than 1% for the estimation of various multicomponent fractions of the permittivity distribution. They are also shown to give improved component fraction estimation compared to a well known direct ECT method.

Pulvinar thalamic nucleus allows for asynchronous spike propagation through the cortex

PubMed Central

Cortes, Nelson; van Vreeswijk, Carl

2015-01-01

We create two multilayered feedforward networks composed of excitatory and inhibitory integrate-and-fire neurons in the balanced state to investigate the role of cortico-pulvino-cortical connections. The first network consists of ten feedforward levels where a Poisson spike train with varying firing rate is applied as an input in layer one. Although the balanced state partially avoids spike synchronization during the transmission, the average firing-rate in the last layer either decays or saturates depending on the feedforward pathway gain. The last layer activity is almost independent of the input even for a carefully chosen intermediate gain. Adding connections to the feedforward pathway by a nine areas Pulvinar structure improves the firing-rate propagation to become almost linear among layers. Incoming strong pulvinar spikes balance the low feedforward gain to have a unit input-output relation in the last layer. Pulvinar neurons evoke a bimodal activity depending on the magnitude input: synchronized spike bursts between 20 and 80 Hz and an asynchronous activity for very both low and high frequency inputs. In the first regime, spikes of last feedforward layer neurons are asynchronous with weak, low frequency, oscillations in the rate. Here, the uncorrelated incoming feedforward pathway washes out the synchronized thalamic bursts. In the second regime, spikes in the whole network are asynchronous. As the number of cortical layers increases, long-range pulvinar connections can link directly two or more cortical stages avoiding their either saturation or gradual activity falling. The Pulvinar acts as a shortcut that supplies the input-output firing-rate relationship of two separated cortical areas without changing the strength of connections in the feedforward pathway. PMID:26042026

A real time neural net estimator of fatigue life

NASA Technical Reports Server (NTRS)

Troudet, T.; Merrill, W.

1990-01-01

A neural net architecture is proposed to estimate, in real-time, the fatigue life of mechanical components, as part of the Intelligent Control System for Reusable Rocket Engines. Arbitrary component loading values were used as input to train a two hidden-layer feedforward neural net to estimate component fatigue damage. The ability of the net to learn, based on a local strain approach, the mapping between load sequence and fatigue damage has been demonstrated for a uniaxial specimen. Because of its demonstrated performance, the neural computation may be extended to complex cases where the loads are biaxial or triaxial, and the geometry of the component is complex (e.g., turbopump blades). The generality of the approach is such that load/damage mappings can be directly extracted from experimental data without requiring any knowledge of the stress/strain profile of the component. In addition, the parallel network architecture allows real-time life calculations even for high frequency vibrations. Owing to its distributed nature, the neural implementation will be robust and reliable, enabling its use in hostile environments such as rocket engines. This neural net estimator of fatigue life is seen as the enabling technology to achieve component life prognosis, and therefore would be an important part of life extending control for reusable rocket engines.

Sharpening of Hierarchical Visual Feature Representations of Blurred Images.

PubMed

Abdelhack, Mohamed; Kamitani, Yukiyasu

2018-01-01

The robustness of the visual system lies in its ability to perceive degraded images. This is achieved through interacting bottom-up, recurrent, and top-down pathways that process the visual input in concordance with stored prior information. The interaction mechanism by which they integrate visual input and prior information is still enigmatic. We present a new approach using deep neural network (DNN) representation to reveal the effects of such integration on degraded visual inputs. We transformed measured human brain activity resulting from viewing blurred images to the hierarchical representation space derived from a feedforward DNN. Transformed representations were found to veer toward the original nonblurred image and away from the blurred stimulus image. This indicated deblurring or sharpening in the neural representation, and possibly in our perception. We anticipate these results will help unravel the interplay mechanism between bottom-up, recurrent, and top-down pathways, leading to more comprehensive models of vision.

Predicting non-linear dynamics by stable local learning in a recurrent spiking neural network.

PubMed

Gilra, Aditya; Gerstner, Wulfram

2017-11-27

The brain needs to predict how the body reacts to motor commands, but how a network of spiking neurons can learn non-linear body dynamics using local, online and stable learning rules is unclear. Here, we present a supervised learning scheme for the feedforward and recurrent connections in a network of heterogeneous spiking neurons. The error in the output is fed back through fixed random connections with a negative gain, causing the network to follow the desired dynamics. The rule for Feedback-based Online Local Learning Of Weights (FOLLOW) is local in the sense that weight changes depend on the presynaptic activity and the error signal projected onto the postsynaptic neuron. We provide examples of learning linear, non-linear and chaotic dynamics, as well as the dynamics of a two-link arm. Under reasonable approximations, we show, using the Lyapunov method, that FOLLOW learning is uniformly stable, with the error going to zero asymptotically.

Predicting non-linear dynamics by stable local learning in a recurrent spiking neural network

PubMed Central

Gerstner, Wulfram

2017-01-01

The brain needs to predict how the body reacts to motor commands, but how a network of spiking neurons can learn non-linear body dynamics using local, online and stable learning rules is unclear. Here, we present a supervised learning scheme for the feedforward and recurrent connections in a network of heterogeneous spiking neurons. The error in the output is fed back through fixed random connections with a negative gain, causing the network to follow the desired dynamics. The rule for Feedback-based Online Local Learning Of Weights (FOLLOW) is local in the sense that weight changes depend on the presynaptic activity and the error signal projected onto the postsynaptic neuron. We provide examples of learning linear, non-linear and chaotic dynamics, as well as the dynamics of a two-link arm. Under reasonable approximations, we show, using the Lyapunov method, that FOLLOW learning is uniformly stable, with the error going to zero asymptotically. PMID:29173280

Top-Down Modulation on Perceptual Decision with Balanced Inhibition through Feedforward and Feedback Inhibitory Neurons

PubMed Central

Wang, Cheng-Te; Lee, Chung-Ting; Wang, Xiao-Jing; Lo, Chung-Chuan

2013-01-01

Recent physiological studies have shown that neurons in various regions of the central nervous systems continuously receive noisy excitatory and inhibitory synaptic inputs in a balanced and covaried fashion. While this balanced synaptic input (BSI) is typically described in terms of maintaining the stability of neural circuits, a number of experimental and theoretical studies have suggested that BSI plays a proactive role in brain functions such as top-down modulation for executive control. Two issues have remained unclear in this picture. First, given the noisy nature of neuronal activities in neural circuits, how do the modulatory effects change if the top-down control implements BSI with different ratios between inhibition and excitation? Second, how is a top-down BSI realized via only excitatory long-range projections in the neocortex? To address the first issue, we systematically tested how the inhibition/excitation ratio affects the accuracy and reaction times of a spiking neural circuit model of perceptual decision. We defined an energy function to characterize the network dynamics, and found that different ratios modulate the energy function of the circuit differently and form two distinct functional modes. To address the second issue, we tested BSI with long-distance projection to inhibitory neurons that are either feedforward or feedback, depending on whether these inhibitory neurons do or do not receive inputs from local excitatory cells, respectively. We found that BSI occurs in both cases. Furthermore, when relying on feedback inhibitory neurons, through the recurrent interactions inside the circuit, BSI dynamically and automatically speeds up the decision by gradually reducing its inhibitory component in the course of a trial when a decision process takes too long. PMID:23626812

Top-down modulation on perceptual decision with balanced inhibition through feedforward and feedback inhibitory neurons.

PubMed

Wang, Cheng-Te; Lee, Chung-Ting; Wang, Xiao-Jing; Lo, Chung-Chuan

2013-01-01

Recent physiological studies have shown that neurons in various regions of the central nervous systems continuously receive noisy excitatory and inhibitory synaptic inputs in a balanced and covaried fashion. While this balanced synaptic input (BSI) is typically described in terms of maintaining the stability of neural circuits, a number of experimental and theoretical studies have suggested that BSI plays a proactive role in brain functions such as top-down modulation for executive control. Two issues have remained unclear in this picture. First, given the noisy nature of neuronal activities in neural circuits, how do the modulatory effects change if the top-down control implements BSI with different ratios between inhibition and excitation? Second, how is a top-down BSI realized via only excitatory long-range projections in the neocortex? To address the first issue, we systematically tested how the inhibition/excitation ratio affects the accuracy and reaction times of a spiking neural circuit model of perceptual decision. We defined an energy function to characterize the network dynamics, and found that different ratios modulate the energy function of the circuit differently and form two distinct functional modes. To address the second issue, we tested BSI with long-distance projection to inhibitory neurons that are either feedforward or feedback, depending on whether these inhibitory neurons do or do not receive inputs from local excitatory cells, respectively. We found that BSI occurs in both cases. Furthermore, when relying on feedback inhibitory neurons, through the recurrent interactions inside the circuit, BSI dynamically and automatically speeds up the decision by gradually reducing its inhibitory component in the course of a trial when a decision process takes too long.

A multi-scale hybrid neural network retrieval model for dust storm detection, a study in Asia

NASA Astrophysics Data System (ADS)

Wong, Man Sing; Xiao, Fei; Nichol, Janet; Fung, Jimmy; Kim, Jhoon; Campbell, James; Chan, P. W.

2015-05-01

Dust storms are known to have adverse effects on human health and significant impact on weather, air quality, hydrological cycle, and ecosystem. Atmospheric dust loading is also one of the large uncertainties in global climate modeling, due to its significant impact on the radiation budget and atmospheric stability. Observations of dust storms in humid tropical south China (e.g. Hong Kong), are challenging due to high industrial pollution from the nearby Pearl River Delta region. This study develops a method for dust storm detection by combining ground station observations (PM10 concentration, AERONET data), geostationary satellite images (MTSAT), and numerical weather and climatic forecasting products (WRF/Chem). The method is based on a hybrid neural network (NN) retrieval model for two scales: (i) a NN model for near real-time detection of dust storms at broader regional scale; (ii) a NN model for detailed dust storm mapping for Hong Kong and Taiwan. A feed-forward multilayer perceptron (MLP) NN, trained using back propagation (BP) algorithm, was developed and validated by the k-fold cross validation approach. The accuracy of the near real-time detection MLP-BP network is 96.6%, and the accuracies for the detailed MLP-BP neural network for Hong Kong and Taiwan is 74.8%. This newly automated multi-scale hybrid method can be used to give advance near real-time mapping of dust storms for environmental authorities and the public. It is also beneficial for identifying spatial locations of adverse air quality conditions, and estimates of low visibility associated with dust events for port and airport authorities.

Feedback Regulation and Its Efficiency in Biochemical Networks

NASA Astrophysics Data System (ADS)

Kobayashi, Tetsuya J.; Yokota, Ryo; Aihara, Kazuyuki

2016-03-01

Intracellular biochemical networks fluctuate dynamically due to various internal and external sources of fluctuation. Dissecting the fluctuation into biologically relevant components is important for understanding how a cell controls and harnesses noise and how information is transferred over apparently noisy intracellular networks. While substantial theoretical and experimental advancement on the decomposition of fluctuation was achieved for feedforward networks without any loop, we still lack a theoretical basis that can consistently extend such advancement to feedback networks. The main obstacle that hampers is the circulative propagation of fluctuation by feedback loops. In order to define the relevant quantity for the impact of feedback loops for fluctuation, disentanglement of the causally interlocked influences between the components is required. In addition, we also lack an approach that enables us to infer non-perturbatively the influence of the feedback to fluctuation in the same way as the dual reporter system does in the feedforward networks. In this work, we address these problems by extending the work on the fluctuation decomposition and the dual reporter system. For a single-loop feedback network with two components, we define feedback loop gain as the feedback efficiency that is consistent with the fluctuation decomposition for feedforward networks. Then, we clarify the relation of the feedback efficiency with the fluctuation propagation in an open-looped FF network. Finally, by extending the dual reporter system, we propose a conjugate feedback and feedforward system for estimating the feedback efficiency non-perturbatively only from the statistics of the system.

Matching and selection of a specific subjective experience: conjugate matching and experience.

PubMed

Vimal, Ram Lakhan Pandey

2010-06-01

We incorporate the dual-mode concept in our dual-aspect PE-SE (proto-experience-subjective experience) framework. The two modes are: (1) the non-tilde mode that is the physical (material) and mental aspect of cognition (memory and attention) related feedback signals in a neural-network, which refers to the cognitive nearest past approaching towards present; and (2) the tilde mode that is the material and mental aspect of the feed-forward signals due to external environmental input and internal endogenous input, which pertains to the nearest future approaching towards present and is a entropy-reversed representation of non-tilde mode. Furthermore, one could argue that there are at least five sub-pathways in the stimulus-dependent feed-forward pathway and cognitive feedback pathway for information transfer in the brain dynamics: (i) classical axonal-dendritic neural sub-pathway including electromagnetic information field sub-pathway; (ii) quantum dendritic-dendritic microtubule (MT) (dendritic webs) sub-pathway; (iii) Ca(++)-related astroglial-neural sub-pathway; (iv) (a) the sub-pathway related to extrasynaptic signal transmission between fine distal dendrites of cortical neurons for the local subtle modulation due to voltages created by intradendritic dual-aspect charged surface effects within the Debye layer around endogenous structures such as microtubules (MT) and endoplasmic reticulum (ER) in dendrites, and (b) the sub-pathway related to extracellular volume transmission as fields of neural activity for the global modulation in axonal-dendritic neural sub-pathway; and (v) the sub-pathway related to information transmission via soliton propagation. We propose that: (i) the quantum conjugate matching between experiences in the mental aspect of the tilde mode and that of the non-tilde mode is related more to the mental aspect of the quantum microtubule-dendritic-web and less to that of the non-quantum sub-pathways; and (ii) the classical matching between experiences in the mental aspect of the tilde mode and that of the non-tilde mode is related to the mental aspect of the non-quantum sub-pathways (such as classical axonal-dendritic neural sub-pathway). In both cases, a specific SE is selected when the tilde mode interacts with the non-tilde mode to match for a specific SE, and when the necessary ingredients of SEs (such as the formation of neural networks, wakefulness, re-entry, attention, working memory, and so on) are satisfied. When the conjugate match is made between the two modes, the world-presence (Now) is disclosed. The material aspects in the tilde mode and that in the non-tilde mode are matched to link structure with function, whereas the mental aspects in the tilde mode and that in the non-tilde mode are matched to link experience with structure and function.

An artificial neural network to predict resting energy expenditure in obesity.

PubMed

Disse, Emmanuel; Ledoux, Séverine; Bétry, Cécile; Caussy, Cyrielle; Maitrepierre, Christine; Coupaye, Muriel; Laville, Martine; Simon, Chantal

2017-09-01

The resting energy expenditure (REE) determination is important in nutrition for adequate dietary prescription. The gold standard i.e. indirect calorimetry is not available in clinical settings. Thus, several predictive equations have been developed, but they lack of accuracy in subjects with extreme weight including obese populations. Artificial neural networks (ANN) are useful predictive tools in the area of artificial intelligence, used in numerous clinical fields. The aim of this study was to determine the relevance of ANN in predicting REE in obesity. A Multi-Layer Perceptron (MLP) feed-forward neural network with a back propagation algorithm was created and cross-validated in a cohort of 565 obese subjects (BMI within 30-50 kg m -2 ) with weight, height, sex and age as clinical inputs and REE measured by indirect calorimetry as output. The predictive performances of ANN were compared to those of 23 predictive REE equations in the training set and in two independent sets of 100 and 237 obese subjects for external validation. Among the 23 established prediction equations for REE evaluated, the Harris & Benedict equations recalculated by Roza were the most accurate for the obese population, followed by the USA DRI, Müller and the original Harris & Benedict equations. The final 5-fold cross-validated three-layer 4-3-1 feed-forward back propagation ANN model developed in that study improved precision and accuracy of REE prediction over linear equations (precision = 68.1%, MAPE = 8.6% and RMSPE = 210 kcal/d), independently from BMI subgroups within 30-50 kg m -2 . External validation confirmed the better predictive performances of ANN model (precision = 73% and 65%, MAPE = 7.7% and 8.6%, RMSPE = 187 kcal/d and 200 kcal/d in the 2 independent datasets) for the prediction of REE in obese subjects. We developed and validated an ANN model for the prediction of REE in obese subjects that is more precise and accurate than established REE predictive equations independent from BMI subgroups. For convenient use in clinical settings, we provide a simple ANN-REE calculator available at: https://www.crnh-rhone-alpes.fr/fr/ANN-REE-Calculator. Copyright © 2017 Elsevier Ltd and European Society for Clinical Nutrition and Metabolism. All rights reserved.

Optimal Decision Making in Neural Inhibition Models

ERIC Educational Resources Information Center

van Ravenzwaaij, Don; van der Maas, Han L. J.; Wagenmakers, Eric-Jan

2012-01-01

In their influential "Psychological Review" article, Bogacz, Brown, Moehlis, Holmes, and Cohen (2006) discussed optimal decision making as accomplished by the drift diffusion model (DDM). The authors showed that neural inhibition models, such as the leaky competing accumulator model (LCA) and the feedforward inhibition model (FFI), can mimic the…

A signal-flow-graph approach to on-line gradient calculation.

PubMed

Campolucci, P; Uncini, A; Piazza, F

2000-08-01

A large class of nonlinear dynamic adaptive systems such as dynamic recurrent neural networks can be effectively represented by signal flow graphs (SFGs). By this method, complex systems are described as a general connection of many simple components, each of them implementing a simple one-input, one-output transformation, as in an electrical circuit. Even if graph representations are popular in the neural network community, they are often used for qualitative description rather than for rigorous representation and computational purposes. In this article, a method for both on-line and batch-backward gradient computation of a system output or cost function with respect to system parameters is derived by the SFG representation theory and its known properties. The system can be any causal, in general nonlinear and time-variant, dynamic system represented by an SFG, in particular any feedforward, time-delay, or recurrent neural network. In this work, we use discrete-time notation, but the same theory holds for the continuous-time case. The gradient is obtained in a straightforward way by the analysis of two SFGs, the original one and its adjoint (obtained from the first by simple transformations), without the complex chain rule expansions of derivatives usually employed. This method can be used for sensitivity analysis and for learning both off-line and on-line. On-line learning is particularly important since it is required by many real applications, such as digital signal processing, system identification and control, channel equalization, and predistortion.

Use of neural networks to model complex immunogenetic associations of disease: human leukocyte antigen impact on the progression of human immunodeficiency virus infection.

PubMed

Ioannidis, J P; McQueen, P G; Goedert, J J; Kaslow, R A

1998-03-01

Complex immunogenetic associations of disease involving a large number of gene products are difficult to evaluate with traditional statistical methods and may require complex modeling. The authors evaluated the performance of feed-forward backpropagation neural networks in predicting rapid progression to acquired immunodeficiency syndrome (AIDS) for patients with human immunodeficiency virus (HIV) infection on the basis of major histocompatibility complex variables. Networks were trained on data from patients from the Multicenter AIDS Cohort Study (n = 139) and then validated on patients from the DC Gay cohort (n = 102). The outcome of interest was rapid disease progression, defined as progression to AIDS in <6 years from seroconversion. Human leukocyte antigen (HLA) variables were selected as network inputs with multivariate regression and a previously described algorithm selecting markers with extreme point estimates for progression risk. Network performance was compared with that of logistic regression. Networks with 15 HLA inputs and a single hidden layer of five nodes achieved a sensitivity of 87.5% and specificity of 95.6% in the training set, vs. 77.0% and 76.9%, respectively, achieved by logistic regression. When validated on the DC Gay cohort, networks averaged a sensitivity of 59.1% and specificity of 74.3%, vs. 53.1% and 61.4%, respectively, for logistic regression. Neural networks offer further support to the notion that HIV disease progression may be dependent on complex interactions between different class I and class II alleles and transporters associated with antigen processing variants. The effect in the current models is of moderate magnitude, and more data as well as other host and pathogen variables may need to be considered to improve the performance of the models. Artificial intelligence methods may complement linear statistical methods for evaluating immunogenetic associations of disease.

Neural correlates of heart-focused interoception: a functional magnetic resonance imaging meta-analysis

PubMed Central

2016-01-01

Interoception is the ability to perceive one's internal body state including visceral sensations. Heart-focused interoception has received particular attention, in part due to a readily available task for behavioural assessment, but also due to accumulating evidence for a significant role in emotional experience, decision-making and clinical disorders such as anxiety and depression. Improved understanding of the underlying neural correlates is important to promote development of anatomical-functional models and suitable intervention strategies. In the present meta-analysis, nine studies reporting neural activity associated with interoceptive attentiveness (i.e. focused attention to a particular interoceptive signal for a given time interval) to one's heartbeat were submitted to a multilevel kernel density analysis. The findings corroborated an extended network associated with heart-focused interoceptive attentiveness including the posterior right and left insula, right claustrum, precentral gyrus and medial frontal gyrus. Right-hemispheric dominance emphasizes non-verbal information processing with the posterior insula presumably serving as the major gateway for cardioception. Prefrontal neural activity may reflect both top-down attention deployment and processing of feed-forward cardioceptive information, possibly orchestrated via the claustrum. This article is part of the themed issue ‘Interoception beyond homeostasis: affect, cognition and mental health’. PMID:28080975

Artificial neural network-aided image analysis system for cell counting.

PubMed

Sjöström, P J; Frydel, B R; Wahlberg, L U

1999-05-01

In histological preparations containing debris and synthetic materials, it is difficult to automate cell counting using standard image analysis tools, i.e., systems that rely on boundary contours, histogram thresholding, etc. In an attempt to mimic manual cell recognition, an automated cell counter was constructed using a combination of artificial intelligence and standard image analysis methods. Artificial neural network (ANN) methods were applied on digitized microscopy fields without pre-ANN feature extraction. A three-layer feed-forward network with extensive weight sharing in the first hidden layer was employed and trained on 1,830 examples using the error back-propagation algorithm on a Power Macintosh 7300/180 desktop computer. The optimal number of hidden neurons was determined and the trained system was validated by comparison with blinded human counts. System performance at 50x and lO0x magnification was evaluated. The correlation index at 100x magnification neared person-to-person variability, while 50x magnification was not useful. The system was approximately six times faster than an experienced human. ANN-based automated cell counting in noisy histological preparations is feasible. Consistent histology and computer power are crucial for system performance. The system provides several benefits, such as speed of analysis and consistency, and frees up personnel for other tasks.