Robust design of feedback feed-forward iterative learning control based on 2D system theory for linear uncertain systems

NASA Astrophysics Data System (ADS)

Li, Zhifu; Hu, Yueming; Li, Di

2016-08-01

For a class of linear discrete-time uncertain systems, a feedback feed-forward iterative learning control (ILC) scheme is proposed, which is comprised of an iterative learning controller and two current iteration feedback controllers. The iterative learning controller is used to improve the performance along the iteration direction and the feedback controllers are used to improve the performance along the time direction. First of all, the uncertain feedback feed-forward ILC system is presented by an uncertain two-dimensional Roesser model system. Then, two robust control schemes are proposed. One can ensure that the feedback feed-forward ILC system is bounded-input bounded-output stable along time direction, and the other can ensure that the feedback feed-forward ILC system is asymptotically stable along time direction. Both schemes can guarantee the system is robust monotonically convergent along the iteration direction. Third, the robust convergent sufficient conditions are given, which contains a linear matrix inequality (LMI). Moreover, the LMI can be used to determine the gain matrix of the feedback feed-forward iterative learning controller. Finally, the simulation results are presented to demonstrate the effectiveness of the proposed schemes.

Learning control system design based on 2-D theory - An application to parallel link manipulator

NASA Technical Reports Server (NTRS)

Geng, Z.; Carroll, R. L.; Lee, J. D.; Haynes, L. H.

1990-01-01

An approach to iterative learning control system design based on two-dimensional system theory is presented. A two-dimensional model for the iterative learning control system which reveals the connections between learning control systems and two-dimensional system theory is established. A learning control algorithm is proposed, and the convergence of learning using this algorithm is guaranteed by two-dimensional stability. The learning algorithm is applied successfully to the trajectory tracking control problem for a parallel link robot manipulator. The excellent performance of this learning algorithm is demonstrated by the computer simulation results.

Model-based reinforcement learning with dimension reduction.

PubMed

Tangkaratt, Voot; Morimoto, Jun; Sugiyama, Masashi

2016-12-01

The goal of reinforcement learning is to learn an optimal policy which controls an agent to acquire the maximum cumulative reward. The model-based reinforcement learning approach learns a transition model of the environment from data, and then derives the optimal policy using the transition model. However, learning an accurate transition model in high-dimensional environments requires a large amount of data which is difficult to obtain. To overcome this difficulty, in this paper, we propose to combine model-based reinforcement learning with the recently developed least-squares conditional entropy (LSCE) method, which simultaneously performs transition model estimation and dimension reduction. We also further extend the proposed method to imitation learning scenarios. The experimental results show that policy search combined with LSCE performs well for high-dimensional control tasks including real humanoid robot control. Copyright © 2016 Elsevier Ltd. All rights reserved.

Fuzzy controller training using particle swarm optimization for nonlinear system control.

PubMed

Karakuzu, Cihan

2008-04-01

This paper proposes and describes an effective utilization of particle swarm optimization (PSO) to train a Takagi-Sugeno (TS)-type fuzzy controller. Performance evaluation of the proposed fuzzy training method using the obtained simulation results is provided with two samples of highly nonlinear systems: a continuous stirred tank reactor (CSTR) and a Van der Pol (VDP) oscillator. The superiority of the proposed learning technique is that there is no need for a partial derivative with respect to the parameter for learning. This fuzzy learning technique is suitable for real-time implementation, especially if the system model is unknown and a supervised training cannot be run. In this study, all parameters of the controller are optimized with PSO in order to prove that a fuzzy controller trained by PSO exhibits a good control performance.

Adaptive categorization of ART networks in robot behavior learning using game-theoretic formulation.

PubMed

Fung, Wai-keung; Liu, Yun-hui

2003-12-01

Adaptive Resonance Theory (ART) networks are employed in robot behavior learning. Two of the difficulties in online robot behavior learning, namely, (1) exponential memory increases with time, (2) difficulty for operators to specify learning tasks accuracy and control learning attention before learning. In order to remedy the aforementioned difficulties, an adaptive categorization mechanism is introduced in ART networks for perceptual and action patterns categorization in this paper. A game-theoretic formulation of adaptive categorization for ART networks is proposed for vigilance parameter adaptation for category size control on the categories formed. The proposed vigilance parameter update rule can help improving categorization performance in the aspect of category number stability and solve the problem of selecting initial vigilance parameter prior to pattern categorization in traditional ART networks. Behavior learning using physical robot is conducted to demonstrate the effectiveness of the proposed adaptive categorization mechanism in ART networks.

Experimental Verification of Electric Drive Technologies Based on Artificial Intelligence Tools

NASA Technical Reports Server (NTRS)

Rubaai, Ahmed; Ricketts, Daniel; Kotaru, Raj; Thomas, Robert; Noga, Donald F. (Technical Monitor); Kankam, Mark D. (Technical Monitor)

2000-01-01

In this report, a fully integrated prototype of a flight servo control system is successfully developed and implemented using brushless dc motors. The control system is developed by the fuzzy logic theory, and implemented with a multilayer neural network. First, a neural network-based architecture is introduced for fuzzy logic control. The characteristic rules and their membership functions of fuzzy systems are represented as the processing nodes in the neural network structure. The network structure and the parameter learning are performed simultaneously and online in the fuzzy-neural network system. The structure learning is based on the partition of input space. The parameter learning is based on the supervised gradient decent method, using a delta adaptation law. Using experimental setup, the performance of the proposed control system is evaluated under various operating conditions. Test results are presented and discussed in the report. The proposed learning control system has several advantages, namely, simple structure and learning capability, robustness and high tracking performance and few nodes at hidden layers. In comparison with the PI controller, the proposed fuzzy-neural network system can yield a better dynamic performance with shorter settling time, and without overshoot. Experimental results have shown that the proposed control system is adaptive and robust in responding to a wide range of operating conditions. In summary, the goal of this study is to design and implement-advanced servosystems to actuate control surfaces for flight vehicles, namely, aircraft and helicopters, missiles and interceptors, and mini- and micro-air vehicles.

An on-line modified least-mean-square algorithm for training neurofuzzy controllers.

PubMed

Tan, Woei Wan

2007-04-01

The problem hindering the use of data-driven modelling methods for training controllers on-line is the lack of control over the amount by which the plant is excited. As the operating schedule determines the information available on-line, the knowledge of the process may degrade if the setpoint remains constant for an extended period. This paper proposes an identification algorithm that alleviates "learning interference" by incorporating fuzzy theory into the normalized least-mean-square update rule. The ability of the proposed methodology to achieve faster learning is examined by employing the algorithm to train a neurofuzzy feedforward controller for controlling a liquid level process. Since the proposed identification strategy has similarities with the normalized least-mean-square update rule and the recursive least-square estimator, the on-line learning rates of these algorithms are also compared.

An e-Learning System with MR for Experiments Involving Circuit Construction to Control a Robot

ERIC Educational Resources Information Center

Takemura, Atsushi

2016-01-01

This paper proposes a novel e-Learning system for technological experiments involving electronic circuit-construction and controlling robot motion that are necessary in the field of technology. The proposed system performs automated recognition of circuit images transmitted from individual learners and automatically supplies the learner with…

Learning and Control Model of the Arm for Loading

NASA Astrophysics Data System (ADS)

Kim, Kyoungsik; Kambara, Hiroyuki; Shin, Duk; Koike, Yasuharu

We propose a learning and control model of the arm for a loading task in which an object is loaded onto one hand with the other hand, in the sagittal plane. Postural control during object interactions provides important points to motor control theories in terms of how humans handle dynamics changes and use the information of prediction and sensory feedback. For the learning and control model, we coupled a feedback-error-learning scheme with an Actor-Critic method used as a feedback controller. To overcome sensory delays, a feedforward dynamics model (FDM) was used in the sensory feedback path. We tested the proposed model in simulation using a two-joint arm with six muscles, each with time delays in muscle force generation. By applying the proposed model to the loading task, we showed that motor commands started increasing, before an object was loaded on, to stabilize arm posture. We also found that the FDM contributes to the stabilization by predicting how the hand changes based on contexts of the object and efferent signals. For comparison with other computational models, we present the simulation results of a minimum-variance model.

Terminal iterative learning control based station stop control of a train

NASA Astrophysics Data System (ADS)

Hou, Zhongsheng; Wang, Yi; Yin, Chenkun; Tang, Tao

2011-07-01

The terminal iterative learning control (TILC) method is introduced for the first time into the field of train station stop control and three TILC-based algorithms are proposed in this study. The TILC-based train station stop control approach utilises the terminal stop position error in previous braking process to update the current control profile. The initial braking position, or the braking force, or their combination is chosen as the control input, and corresponding learning law is developed. The terminal stop position error of each algorithm is guaranteed to converge to a small region related with the initial offset of braking position with rigorous analysis. The validity of the proposed algorithms is verified by illustrative numerical examples.

Data-driven model reference control of MIMO vertical tank systems with model-free VRFT and Q-Learning.

PubMed

Radac, Mircea-Bogdan; Precup, Radu-Emil; Roman, Raul-Cristian

2018-02-01

This paper proposes a combined Virtual Reference Feedback Tuning-Q-learning model-free control approach, which tunes nonlinear static state feedback controllers to achieve output model reference tracking in an optimal control framework. The novel iterative Batch Fitted Q-learning strategy uses two neural networks to represent the value function (critic) and the controller (actor), and it is referred to as a mixed Virtual Reference Feedback Tuning-Batch Fitted Q-learning approach. Learning convergence of the Q-learning schemes generally depends, among other settings, on the efficient exploration of the state-action space. Handcrafting test signals for efficient exploration is difficult even for input-output stable unknown processes. Virtual Reference Feedback Tuning can ensure an initial stabilizing controller to be learned from few input-output data and it can be next used to collect substantially more input-state data in a controlled mode, in a constrained environment, by compensating the process dynamics. This data is used to learn significantly superior nonlinear state feedback neural networks controllers for model reference tracking, using the proposed Batch Fitted Q-learning iterative tuning strategy, motivating the original combination of the two techniques. The mixed Virtual Reference Feedback Tuning-Batch Fitted Q-learning approach is experimentally validated for water level control of a multi input-multi output nonlinear constrained coupled two-tank system. Discussions on the observed control behavior are offered. Copyright © 2018 ISA. Published by Elsevier Ltd. All rights reserved.

Adaptive filter design using recurrent cerebellar model articulation controller.

PubMed

Lin, Chih-Min; Chen, Li-Yang; Yeung, Daniel S

2010-07-01

A novel adaptive filter is proposed using a recurrent cerebellar-model-articulation-controller (CMAC). The proposed locally recurrent globally feedforward recurrent CMAC (RCMAC) has favorable properties of small size, good generalization, rapid learning, and dynamic response, thus it is more suitable for high-speed signal processing. To provide fast training, an efficient parameter learning algorithm based on the normalized gradient descent method is presented, in which the learning rates are on-line adapted. Then the Lyapunov function is utilized to derive the conditions of the adaptive learning rates, so the stability of the filtering error can be guaranteed. To demonstrate the performance of the proposed adaptive RCMAC filter, it is applied to a nonlinear channel equalization system and an adaptive noise cancelation system. The advantages of the proposed filter over other adaptive filters are verified through simulations.

Consensus-based distributed cooperative learning from closed-loop neural control systems.

PubMed

Chen, Weisheng; Hua, Shaoyong; Zhang, Huaguang

2015-02-01

In this paper, the neural tracking problem is addressed for a group of uncertain nonlinear systems where the system structures are identical but the reference signals are different. This paper focuses on studying the learning capability of neural networks (NNs) during the control process. First, we propose a novel control scheme called distributed cooperative learning (DCL) control scheme, by establishing the communication topology among adaptive laws of NN weights to share their learned knowledge online. It is further proved that if the communication topology is undirected and connected, all estimated weights of NNs can converge to small neighborhoods around their optimal values over a domain consisting of the union of all state orbits. Second, as a corollary it is shown that the conclusion on the deterministic learning still holds in the decentralized adaptive neural control scheme where, however, the estimated weights of NNs just converge to small neighborhoods of the optimal values along their own state orbits. Thus, the learned controllers obtained by DCL scheme have the better generalization capability than ones obtained by decentralized learning method. A simulation example is provided to verify the effectiveness and advantages of the control schemes proposed in this paper.

Genetic reinforcement learning through symbiotic evolution for fuzzy controller design.

PubMed

Juang, C F; Lin, J Y; Lin, C T

2000-01-01

An efficient genetic reinforcement learning algorithm for designing fuzzy controllers is proposed in this paper. The genetic algorithm (GA) adopted in this paper is based upon symbiotic evolution which, when applied to fuzzy controller design, complements the local mapping property of a fuzzy rule. Using this Symbiotic-Evolution-based Fuzzy Controller (SEFC) design method, the number of control trials, as well as consumed CPU time, are considerably reduced when compared to traditional GA-based fuzzy controller design methods and other types of genetic reinforcement learning schemes. Moreover, unlike traditional fuzzy controllers, which partition the input space into a grid, SEFC partitions the input space in a flexible way, thus creating fewer fuzzy rules. In SEFC, different types of fuzzy rules whose consequent parts are singletons, fuzzy sets, or linear equations (TSK-type fuzzy rules) are allowed. Further, the free parameters (e.g., centers and widths of membership functions) and fuzzy rules are all tuned automatically. For the TSK-type fuzzy rule especially, which put the proposed learning algorithm in use, only the significant input variables are selected to participate in the consequent of a rule. The proposed SEFC design method has been applied to different simulated control problems, including the cart-pole balancing system, a magnetic levitation system, and a water bath temperature control system. The proposed SEFC has been verified to be efficient and superior from these control problems, and from comparisons with some traditional GA-based fuzzy systems.

Intelligent control based on fuzzy logic and neural net theory

NASA Technical Reports Server (NTRS)

Lee, Chuen-Chien

1991-01-01

In the conception and design of intelligent systems, one promising direction involves the use of fuzzy logic and neural network theory to enhance such systems' capability to learn from experience and adapt to changes in an environment of uncertainty and imprecision. Here, an intelligent control scheme is explored by integrating these multidisciplinary techniques. A self-learning system is proposed as an intelligent controller for dynamical processes, employing a control policy which evolves and improves automatically. One key component of the intelligent system is a fuzzy logic-based system which emulates human decision making behavior. It is shown that the system can solve a fairly difficult control learning problem. Simulation results demonstrate that improved learning performance can be achieved in relation to previously described systems employing bang-bang control. The proposed system is relatively insensitive to variations in the parameters of the system environment.

Approximate reasoning-based learning and control for proximity operations and docking in space

NASA Technical Reports Server (NTRS)

Berenji, Hamid R.; Jani, Yashvant; Lea, Robert N.

1991-01-01

A recently proposed hybrid-neutral-network and fuzzy-logic-control architecture is applied to a fuzzy logic controller developed for attitude control of the Space Shuttle. A model using reinforcement learning and learning from past experience for fine-tuning its knowledge base is proposed. Two main components of this approximate reasoning-based intelligent control (ARIC) model - an action-state evaluation network and action selection network are described as well as the Space Shuttle attitude controller. An ARIC model for the controller is presented, and it is noted that the input layer in each network includes three nodes representing the angle error, angle error rate, and bias node. Preliminary results indicate that the controller can hold the pitch rate within its desired deadband and starts to use the jets at about 500 sec in the run.

Adaptive control of nonlinear system using online error minimum neural networks.

PubMed

Jia, Chao; Li, Xiaoli; Wang, Kang; Ding, Dawei

2016-11-01

In this paper, a new learning algorithm named OEM-ELM (Online Error Minimized-ELM) is proposed based on ELM (Extreme Learning Machine) neural network algorithm and the spreading of its main structure. The core idea of this OEM-ELM algorithm is: online learning, evaluation of network performance, and increasing of the number of hidden nodes. It combines the advantages of OS-ELM and EM-ELM, which can improve the capability of identification and avoid the redundancy of networks. The adaptive control based on the proposed algorithm OEM-ELM is set up which has stronger adaptive capability to the change of environment. The adaptive control of chemical process Continuous Stirred Tank Reactor (CSTR) is also given for application. The simulation results show that the proposed algorithm with respect to the traditional ELM algorithm can avoid network redundancy and improve the control performance greatly. Copyright © 2016 ISA. Published by Elsevier Ltd. All rights reserved.

Impedance learning for robotic contact tasks using natural actor-critic algorithm.

PubMed

Kim, Byungchan; Park, Jooyoung; Park, Shinsuk; Kang, Sungchul

2010-04-01

Compared with their robotic counterparts, humans excel at various tasks by using their ability to adaptively modulate arm impedance parameters. This ability allows us to successfully perform contact tasks even in uncertain environments. This paper considers a learning strategy of motor skill for robotic contact tasks based on a human motor control theory and machine learning schemes. Our robot learning method employs impedance control based on the equilibrium point control theory and reinforcement learning to determine the impedance parameters for contact tasks. A recursive least-square filter-based episodic natural actor-critic algorithm is used to find the optimal impedance parameters. The effectiveness of the proposed method was tested through dynamic simulations of various contact tasks. The simulation results demonstrated that the proposed method optimizes the performance of the contact tasks in uncertain conditions of the environment.

Remote Learning for the Manipulation and Control of Robotic Cells

ERIC Educational Resources Information Center

Goldstain, Ofir; Ben-Gal, Irad; Bukchin, Yossi

2007-01-01

This work proposes an approach to remote learning of robotic cells based on internet and simulation tools. The proposed approach, which integrates remote-learning and tele-operation into a generic scheme, is designed to enable students and developers to set-up and manipulate a robotic cell remotely. Its implementation is based on a dedicated…

The Perplexing Role of Learner Control in E-Learning: Will Learning and Transfer Benefit or Suffer?

ERIC Educational Resources Information Center

Granger, Benjamin P.; Levine, Edward L.

2010-01-01

The appeal of e-learning is not surprising given its many proposed advantages (e.g. flexibility, responsiveness to trainees' individual needs, potential cost-effectiveness). However, as pointed out by Ruel et al., academic research in support of the proposed advantages of electronic human resource management systems is scarce. Although this is…

Fuzzy self-learning control for magnetic servo system

NASA Technical Reports Server (NTRS)

Tarn, J. H.; Kuo, L. T.; Juang, K. Y.; Lin, C. E.

1994-01-01

It is known that an effective control system is the key condition for successful implementation of high-performance magnetic servo systems. Major issues to design such control systems are nonlinearity; unmodeled dynamics, such as secondary effects for copper resistance, stray fields, and saturation; and that disturbance rejection for the load effect reacts directly on the servo system without transmission elements. One typical approach to design control systems under these conditions is a special type of nonlinear feedback called gain scheduling. It accommodates linear regulators whose parameters are changed as a function of operating conditions in a preprogrammed way. In this paper, an on-line learning fuzzy control strategy is proposed. To inherit the wealth of linear control design, the relations between linear feedback and fuzzy logic controllers have been established. The exercise of engineering axioms of linear control design is thus transformed into tuning of appropriate fuzzy parameters. Furthermore, fuzzy logic control brings the domain of candidate control laws from linear into nonlinear, and brings new prospects into design of the local controllers. On the other hand, a self-learning scheme is utilized to automatically tune the fuzzy rule base. It is based on network learning infrastructure; statistical approximation to assign credit; animal learning method to update the reinforcement map with a fast learning rate; and temporal difference predictive scheme to optimize the control laws. Different from supervised and statistical unsupervised learning schemes, the proposed method learns on-line from past experience and information from the process and forms a rule base of an FLC system from randomly assigned initial control rules.

A reinforcement learning-based architecture for fuzzy logic control

NASA Technical Reports Server (NTRS)

Berenji, Hamid R.

1992-01-01

This paper introduces a new method for learning to refine a rule-based fuzzy logic controller. A reinforcement learning technique is used in conjunction with a multilayer neural network model of a fuzzy controller. The approximate reasoning based intelligent control (ARIC) architecture proposed here learns by updating its prediction of the physical system's behavior and fine tunes a control knowledge base. Its theory is related to Sutton's temporal difference (TD) method. Because ARIC has the advantage of using the control knowledge of an experienced operator and fine tuning it through the process of learning, it learns faster than systems that train networks from scratch. The approach is applied to a cart-pole balancing system.

Magnetic induction of hyperthermia by a modified self-learning fuzzy temperature controller

NASA Astrophysics Data System (ADS)

Wang, Wei-Cheng; Tai, Cheng-Chi

2017-07-01

The aim of this study involved developing a temperature controller for magnetic induction hyperthermia (MIH). A closed-loop controller was applied to track a reference model to guarantee a desired temperature response. The MIH system generated an alternating magnetic field to heat a high magnetic permeability material. This wireless induction heating had few side effects when it was extensively applied to cancer treatment. The effects of hyperthermia strongly depend on the precise control of temperature. However, during the treatment process, the control performance is degraded due to severe perturbations and parameter variations. In this study, a modified self-learning fuzzy logic controller (SLFLC) with a gain tuning mechanism was implemented to obtain high control performance in a wide range of treatment situations. This implementation was performed by appropriately altering the output scaling factor of a fuzzy inverse model to adjust the control rules. In this study, the proposed SLFLC was compared to the classical self-tuning fuzzy logic controller and fuzzy model reference learning control. Additionally, the proposed SLFLC was verified by conducting in vitro experiments with porcine liver. The experimental results indicated that the proposed controller showed greater robustness and excellent adaptability with respect to the temperature control of the MIH system.

Cerebellar supervised learning revisited: biophysical modeling and degrees-of-freedom control.

PubMed

Kawato, Mitsuo; Kuroda, Shinya; Schweighofer, Nicolas

2011-10-01

The biophysical models of spike-timing-dependent plasticity have explored dynamics with molecular basis for such computational concepts as coincidence detection, synaptic eligibility trace, and Hebbian learning. They overall support different learning algorithms in different brain areas, especially supervised learning in the cerebellum. Because a single spine is physically very small, chemical reactions at it are essentially stochastic, and thus sensitivity-longevity dilemma exists in the synaptic memory. Here, the cascade of excitable and bistable dynamics is proposed to overcome this difficulty. All kinds of learning algorithms in different brain regions confront with difficult generalization problems. For resolution of this issue, the control of the degrees-of-freedom can be realized by changing synchronicity of neural firing. Especially, for cerebellar supervised learning, the triangle closed-loop circuit consisting of Purkinje cells, the inferior olive nucleus, and the cerebellar nucleus is proposed as a circuit to optimally control synchronous firing and degrees-of-freedom in learning. Copyright © 2011 Elsevier Ltd. All rights reserved.

Motor-Skill Learning in an Insect Inspired Neuro-Computational Control System

PubMed Central

Arena, Eleonora; Arena, Paolo; Strauss, Roland; Patané, Luca

2017-01-01

In nature, insects show impressive adaptation and learning capabilities. The proposed computational model takes inspiration from specific structures of the insect brain: after proposing key hypotheses on the direct involvement of the mushroom bodies (MBs) and on their neural organization, we developed a new architecture for motor learning to be applied in insect-like walking robots. The proposed model is a nonlinear control system based on spiking neurons. MBs are modeled as a nonlinear recurrent spiking neural network (SNN) with novel characteristics, able to memorize time evolutions of key parameters of the neural motor controller, so that existing motor primitives can be improved. The adopted control scheme enables the structure to efficiently cope with goal-oriented behavioral motor tasks. Here, a six-legged structure, showing a steady-state exponentially stable locomotion pattern, is exposed to the need of learning new motor skills: moving through the environment, the structure is able to modulate motor commands and implements an obstacle climbing procedure. Experimental results on a simulated hexapod robot are reported; they are obtained in a dynamic simulation environment and the robot mimicks the structures of Drosophila melanogaster. PMID:28337138

Resonant Cholinergic Dynamics in Cognitive and Motor Decision-Making: Attention, Category Learning, and Choice in Neocortex, Superior Colliculus, and Optic Tectum.

PubMed

Grossberg, Stephen; Palma, Jesse; Versace, Massimiliano

2015-01-01

Freely behaving organisms need to rapidly calibrate their perceptual, cognitive, and motor decisions based on continuously changing environmental conditions. These plastic changes include sharpening or broadening of cognitive and motor attention and learning to match the behavioral demands that are imposed by changing environmental statistics. This article proposes that a shared circuit design for such flexible decision-making is used in specific cognitive and motor circuits, and that both types of circuits use acetylcholine to modulate choice selectivity. Such task-sensitive control is proposed to control thalamocortical choice of the critical features that are cognitively attended and that are incorporated through learning into prototypes of visual recognition categories. A cholinergically-modulated process of vigilance control determines if a recognition category and its attended features are abstract (low vigilance) or concrete (high vigilance). Homologous neural mechanisms of cholinergic modulation are proposed to focus attention and learn a multimodal map within the deeper layers of superior colliculus. This map enables visual, auditory, and planned movement commands to compete for attention, leading to selection of a winning position that controls where the next saccadic eye movement will go. Such map learning may be viewed as a kind of attentive motor category learning. The article hereby explicates a link between attention, learning, and cholinergic modulation during decision making within both cognitive and motor systems. Homologs between the mammalian superior colliculus and the avian optic tectum lead to predictions about how multimodal map learning may occur in the mammalian and avian brain and how such learning may be modulated by acetycholine.

Resonant Cholinergic Dynamics in Cognitive and Motor Decision-Making: Attention, Category Learning, and Choice in Neocortex, Superior Colliculus, and Optic Tectum

PubMed Central

Grossberg, Stephen; Palma, Jesse; Versace, Massimiliano

2016-01-01

Freely behaving organisms need to rapidly calibrate their perceptual, cognitive, and motor decisions based on continuously changing environmental conditions. These plastic changes include sharpening or broadening of cognitive and motor attention and learning to match the behavioral demands that are imposed by changing environmental statistics. This article proposes that a shared circuit design for such flexible decision-making is used in specific cognitive and motor circuits, and that both types of circuits use acetylcholine to modulate choice selectivity. Such task-sensitive control is proposed to control thalamocortical choice of the critical features that are cognitively attended and that are incorporated through learning into prototypes of visual recognition categories. A cholinergically-modulated process of vigilance control determines if a recognition category and its attended features are abstract (low vigilance) or concrete (high vigilance). Homologous neural mechanisms of cholinergic modulation are proposed to focus attention and learn a multimodal map within the deeper layers of superior colliculus. This map enables visual, auditory, and planned movement commands to compete for attention, leading to selection of a winning position that controls where the next saccadic eye movement will go. Such map learning may be viewed as a kind of attentive motor category learning. The article hereby explicates a link between attention, learning, and cholinergic modulation during decision making within both cognitive and motor systems. Homologs between the mammalian superior colliculus and the avian optic tectum lead to predictions about how multimodal map learning may occur in the mammalian and avian brain and how such learning may be modulated by acetycholine. PMID:26834535

Proposal of a Framework for Internet Based Licensing of Learning Objects

ERIC Educational Resources Information Center

Santos, Osvaldo A.; Ramos, Fernando M. S.

2004-01-01

This paper presents a proposal of a framework whose main objective is to manage the delivery and rendering of learning objects in a digital rights controlled environment. The framework is based on a digital licensing scheme that requires each learning object to have the proper license in order to be rendered by a trusted player. A conceptual model…

Version Control in Project-Based Learning

ERIC Educational Resources Information Center

Milentijevic, Ivan; Ciric, Vladimir; Vojinovic, Oliver

2008-01-01

This paper deals with the development of a generalized model for version control systems application as a support in a range of project-based learning methods. The model is given as UML sequence diagram and described in detail. The proposed model encompasses a wide range of different project-based learning approaches by assigning a supervisory…

Model-based iterative learning control of Parkinsonian state in thalamic relay neuron

NASA Astrophysics Data System (ADS)

Liu, Chen; Wang, Jiang; Li, Huiyan; Xue, Zhiqin; Deng, Bin; Wei, Xile

2014-09-01

Although the beneficial effects of chronic deep brain stimulation on Parkinson's disease motor symptoms are now largely confirmed, the underlying mechanisms behind deep brain stimulation remain unclear and under debate. Hence, the selection of stimulation parameters is full of challenges. Additionally, due to the complexity of neural system, together with omnipresent noises, the accurate model of thalamic relay neuron is unknown. Thus, the iterative learning control of the thalamic relay neuron's Parkinsonian state based on various variables is presented. Combining the iterative learning control with typical proportional-integral control algorithm, a novel and efficient control strategy is proposed, which does not require any particular knowledge on the detailed physiological characteristics of cortico-basal ganglia-thalamocortical loop and can automatically adjust the stimulation parameters. Simulation results demonstrate the feasibility of the proposed control strategy to restore the fidelity of thalamic relay in the Parkinsonian condition. Furthermore, through changing the important parameter—the maximum ionic conductance densities of low-threshold calcium current, the dominant characteristic of the proposed method which is independent of the accurate model can be further verified.

Probabilistic dual heuristic programming-based adaptive critic

NASA Astrophysics Data System (ADS)

Herzallah, Randa

2010-02-01

Adaptive critic (AC) methods have common roots as generalisations of dynamic programming for neural reinforcement learning approaches. Since they approximate the dynamic programming solutions, they are potentially suitable for learning in noisy, non-linear and non-stationary environments. In this study, a novel probabilistic dual heuristic programming (DHP)-based AC controller is proposed. Distinct to current approaches, the proposed probabilistic (DHP) AC method takes uncertainties of forward model and inverse controller into consideration. Therefore, it is suitable for deterministic and stochastic control problems characterised by functional uncertainty. Theoretical development of the proposed method is validated by analytically evaluating the correct value of the cost function which satisfies the Bellman equation in a linear quadratic control problem. The target value of the probabilistic critic network is then calculated and shown to be equal to the analytically derived correct value. Full derivation of the Riccati solution for this non-standard stochastic linear quadratic control problem is also provided. Moreover, the performance of the proposed probabilistic controller is demonstrated on linear and non-linear control examples.

An architecture for designing fuzzy logic controllers using neural networks

NASA Technical Reports Server (NTRS)

Berenji, Hamid R.

1991-01-01

Described here is an architecture for designing fuzzy controllers through a hierarchical process of control rule acquisition and by using special classes of neural network learning techniques. A new method for learning to refine a fuzzy logic controller is introduced. A reinforcement learning technique is used in conjunction with a multi-layer neural network model of a fuzzy controller. The model learns by updating its prediction of the plant's behavior and is related to the Sutton's Temporal Difference (TD) method. The method proposed here has the advantage of using the control knowledge of an experienced operator and fine-tuning it through the process of learning. The approach is applied to a cart-pole balancing system.

Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting.

PubMed

Coop, Robert; Mishtal, Aaron; Arel, Itamar

2013-10-01

Catastrophic forgetting is a well-studied attribute of most parameterized supervised learning systems. A variation of this phenomenon, in the context of feedforward neural networks, arises when nonstationary inputs lead to loss of previously learned mappings. The majority of the schemes proposed in the literature for mitigating catastrophic forgetting were not data driven and did not scale well. We introduce the fixed expansion layer (FEL) feedforward neural network, which embeds a sparsely encoding hidden layer to help mitigate forgetting of prior learned representations. In addition, we investigate a novel framework for training ensembles of FEL networks, based on exploiting an information-theoretic measure of diversity between FEL learners, to further control undesired plasticity. The proposed methodology is demonstrated on a basic classification task, clearly emphasizing its advantages over existing techniques. The architecture proposed can be enhanced to address a range of computational intelligence tasks, such as regression problems and system control.

Model-Free control performance improvement using virtual reference feedback tuning and reinforcement Q-learning

NASA Astrophysics Data System (ADS)

Radac, Mircea-Bogdan; Precup, Radu-Emil; Roman, Raul-Cristian

2017-04-01

This paper proposes the combination of two model-free controller tuning techniques, namely linear virtual reference feedback tuning (VRFT) and nonlinear state-feedback Q-learning, referred to as a new mixed VRFT-Q learning approach. VRFT is first used to find stabilising feedback controller using input-output experimental data from the process in a model reference tracking setting. Reinforcement Q-learning is next applied in the same setting using input-state experimental data collected under perturbed VRFT to ensure good exploration. The Q-learning controller learned with a batch fitted Q iteration algorithm uses two neural networks, one for the Q-function estimator and one for the controller, respectively. The VRFT-Q learning approach is validated on position control of a two-degrees-of-motion open-loop stable multi input-multi output (MIMO) aerodynamic system (AS). Extensive simulations for the two independent control channels of the MIMO AS show that the Q-learning controllers clearly improve performance over the VRFT controllers.

Effects of iterative learning based signal control strategies on macroscopic fundamental diagrams of urban road networks

NASA Astrophysics Data System (ADS)

Yan, Fei; Tian, Fuli; Shi, Zhongke

2016-10-01

Urban traffic flows are inherently repeated on a daily or weekly basis. This repeatability can help improve the traffic conditions if it is used properly by the control system. In this paper, we propose a novel iterative learning control (ILC) strategy for traffic signals of urban road networks using the repeatability feature of traffic flow. To improve the control robustness, the ILC strategy is further integrated with an error feedback control law in a complementary manner. Theoretical analysis indicates that the ILC-based traffic signal control methods can guarantee the asymptotic learning convergence, despite the presence of modeling uncertainties and exogenous disturbances. Finally, the impacts of the ILC-based signal control strategies on the network macroscopic fundamental diagram (MFD) are examined. The results show that the proposed ILC-based control strategies can homogenously distribute the network accumulation by controlling the vehicle numbers in each link to the desired levels under different traffic demands, which can result in the network with high capacity and mobility.

An Integrated Framework for Human-Robot Collaborative Manipulation.

PubMed

Sheng, Weihua; Thobbi, Anand; Gu, Ye

2015-10-01

This paper presents an integrated learning framework that enables humanoid robots to perform human-robot collaborative manipulation tasks. Specifically, a table-lifting task performed jointly by a human and a humanoid robot is chosen for validation purpose. The proposed framework is split into two phases: 1) phase I-learning to grasp the table and 2) phase II-learning to perform the manipulation task. An imitation learning approach is proposed for phase I. In phase II, the behavior of the robot is controlled by a combination of two types of controllers: 1) reactive and 2) proactive. The reactive controller lets the robot take a reactive control action to make the table horizontal. The proactive controller lets the robot take proactive actions based on human motion prediction. A measure of confidence of the prediction is also generated by the motion predictor. This confidence measure determines the leader/follower behavior of the robot. Hence, the robot can autonomously switch between the behaviors during the task. Finally, the performance of the human-robot team carrying out the collaborative manipulation task is experimentally evaluated on a platform consisting of a Nao humanoid robot and a Vicon motion capture system. Results show that the proposed framework can enable the robot to carry out the collaborative manipulation task successfully.

Research on intelligent algorithm of electro - hydraulic servo control system

NASA Astrophysics Data System (ADS)

Wang, Yannian; Zhao, Yuhui; Liu, Chengtao

2017-09-01

In order to adapt the nonlinear characteristics of the electro-hydraulic servo control system and the influence of complex interference in the industrial field, using a fuzzy PID switching learning algorithm is proposed and a fuzzy PID switching learning controller is designed and applied in the electro-hydraulic servo controller. The designed controller not only combines the advantages of the fuzzy control and PID control, but also introduces the learning algorithm into the switching function, which makes the learning of the three parameters in the switching function can avoid the instability of the system during the switching between the fuzzy control and PID control algorithms. It also makes the switch between these two control algorithm more smoother than that of the conventional fuzzy PID.

A Learning Framework for Control-Oriented Modeling of Buildings

DOE Office of Scientific and Technical Information (OSTI.GOV)

Rubio-Herrero, Javier; Chandan, Vikas; Siegel, Charles M.

Buildings consume a significant amount of energy worldwide. Several building optimization and control use cases require models of energy consumption which are control oriented, have high predictive capability, imposes minimal data pre-processing requirements, and have the ability to be adapted continuously to account for changing conditions as new data becomes available. Data driven modeling techniques, that have been investigated so far, while promising in the context of buildings, have been unable to simultaneously satisfy all the requirements mentioned above. In this context, deep learning techniques such as Recurrent Neural Networks (RNNs) hold promise, empowered by advanced computational capabilities and bigmore » data opportunities. In this paper, we propose a deep learning based methodology for the development of control oriented models for building energy management and test in on data from a real building. Results show that the proposed methodology outperforms other data driven modeling techniques significantly. We perform a detailed analysis of the proposed methodology along dimensions such as topology, sensitivity, and downsampling. Lastly, we conclude by envisioning a building analytics suite empowered by the proposed deep framework, that can drive several use cases related to building energy management.« less

The algorithm for duration acceleration of repetitive projects considering the learning effect

NASA Astrophysics Data System (ADS)

Chen, Hongtao; Wang, Keke; Du, Yang; Wang, Liwan

2018-03-01

Repetitive project optimization problem is common in project scheduling. Repetitive Scheduling Method (RSM) has many irreplaceable advantages in the field of repetitive projects. As the same or similar work is repeated, the proficiency of workers will be correspondingly low to high, and workers will gain experience and improve the efficiency of operations. This is learning effect. Learning effect is one of the important factors affecting the optimization results in repetitive project scheduling. This paper analyzes the influence of the learning effect on the controlling path in RSM from two aspects: one is that the learning effect changes the controlling path, the other is that the learning effect doesn't change the controlling path. This paper proposes corresponding methods to accelerate duration for different types of critical activities and proposes the algorithm for duration acceleration based on the learning effect in RSM. And the paper chooses graphical method to identity activities' types and considers the impacts of the learning effect on duration. The method meets the requirement of duration while ensuring the lowest acceleration cost. A concrete bridge construction project is given to verify the effectiveness of the method. The results of this study will help project managers understand the impacts of the learning effect on repetitive projects, and use the learning effect to optimize project scheduling.

A Control Systems Concept Inventory Test Design and Assessment

ERIC Educational Resources Information Center

Bristow, M.; Erkorkmaz, K.; Huissoon, J. P.; Jeon, Soo; Owen, W. S.; Waslander, S. L.; Stubley, G. D.

2012-01-01

Any meaningful initiative to improve the teaching and learning in introductory control systems courses needs a clear test of student conceptual understanding to determine the effectiveness of proposed methods and activities. The authors propose a control systems concept inventory. Development of the inventory was collaborative and iterative. The…

Drive Control Scheme of Electric Power Assisted Wheelchair Based on Neural Network Learning of Human Wheelchair Operation Characteristics

NASA Astrophysics Data System (ADS)

Tanohata, Naoki; Seki, Hirokazu

This paper describes a novel drive control scheme of electric power assisted wheelchairs based on neural network learning of human wheelchair operation characteristics. “Electric power assisted wheelchair” which enhances the drive force of the operator by employing electric motors is expected to be widely used as a mobility support system for elderly and disabled people. However, some handicapped people with paralysis of the muscles of one side of the body cannot maneuver the wheelchair as desired because of the difference in the right and left input force. Therefore, this study proposes a neural network learning system of such human wheelchair operation characteristics and a drive control scheme with variable distribution and assistance ratios. Some driving experiments will be performed to confirm the effectiveness of the proposed control system.

Adaptive learning and control for MIMO system based on adaptive dynamic programming.

PubMed

Fu, Jian; He, Haibo; Zhou, Xinmin

2011-07-01

Adaptive dynamic programming (ADP) is a promising research field for design of intelligent controllers, which can both learn on-the-fly and exhibit optimal behavior. Over the past decades, several generations of ADP design have been proposed in the literature, which have demonstrated many successful applications in various benchmarks and industrial applications. While many of the existing researches focus on multiple-inputs-single-output system with steepest descent search, in this paper we investigate a generalized multiple-input-multiple-output (GMIMO) ADP design for online learning and control, which is more applicable to a wide range of practical real-world applications. Furthermore, an improved weight-updating algorithm based on recursive Levenberg-Marquardt methods is presented and embodied in the GMIMO approach to improve its performance. Finally, we test the performance of this approach based on a practical complex system, namely, the learning and control of the tension and height of the looper system in a hot strip mill. Experimental results demonstrate that the proposed approach can achieve effective and robust performance.

Cogging effect minimization in PMSM position servo system using dual high-order periodic adaptive learning compensation.

PubMed

Luo, Ying; Chen, Yangquan; Pi, Youguo

2010-10-01

Cogging effect which can be treated as a type of position-dependent periodic disturbance, is a serious disadvantage of the permanent magnetic synchronous motor (PMSM). In this paper, based on a simulation system model of PMSM position servo control, the cogging force, viscous friction, and applied load in the real PMSM control system are considered and presented. A dual high-order periodic adaptive learning compensation (DHO-PALC) method is proposed to minimize the cogging effect on the PMSM position and velocity servo system. In this DHO-PALC scheme, more than one previous periods stored information of both the composite tracking error and the estimate of the cogging force is used for the control law updating. Asymptotical stability proof with the proposed DHO-PALC scheme is presented. Simulation is implemented on the PMSM servo system model to illustrate the proposed method. When the constant speed reference is applied, the DHO-PALC can achieve a faster learning convergence speed than the first-order periodic adaptive learning compensation (FO-PALC). Moreover, when the designed reference signal changes periodically, the proposed DHO-PALC can obtain not only faster convergence speed, but also much smaller final error bound than the FO-PALC. Copyright © 2010 ISA. Published by Elsevier Ltd. All rights reserved.

Decomposed fuzzy systems and their application in direct adaptive fuzzy control.

PubMed

Hsueh, Yao-Chu; Su, Shun-Feng; Chen, Ming-Chang

2014-10-01

In this paper, a novel fuzzy structure termed as the decomposed fuzzy system (DFS) is proposed to act as the fuzzy approximator for adaptive fuzzy control systems. The proposed structure is to decompose each fuzzy variable into layers of fuzzy systems, and each layer is to characterize one traditional fuzzy set. Similar to forming fuzzy rules in traditional fuzzy systems, layers from different variables form the so-called component fuzzy systems. DFS is proposed to provide more adjustable parameters to facilitate possible adaptation in fuzzy rules, but without introducing a learning burden. It is because those component fuzzy systems are independent so that it can facilitate minimum distribution learning effects among component fuzzy systems. It can be seen from our experiments that even when the rule number increases, the learning time in terms of cycles is still almost constant. It can also be found that the function approximation capability and learning efficiency of the DFS are much better than that of the traditional fuzzy systems when employed in adaptive fuzzy control systems. Besides, in order to further reduce the computational burden, a simplified DFS is proposed in this paper to satisfy possible real time constraints required in many applications. From our simulation results, it can be seen that the simplified DFS can perform fairly with a more concise decomposition structure.

Control chart pattern recognition using RBF neural network with new training algorithm and practical features.

PubMed

Addeh, Abdoljalil; Khormali, Aminollah; Golilarz, Noorbakhsh Amiri

2018-05-04

The control chart patterns are the most commonly used statistical process control (SPC) tools to monitor process changes. When a control chart produces an out-of-control signal, this means that the process has been changed. In this study, a new method based on optimized radial basis function neural network (RBFNN) is proposed for control chart patterns (CCPs) recognition. The proposed method consists of four main modules: feature extraction, feature selection, classification and learning algorithm. In the feature extraction module, shape and statistical features are used. Recently, various shape and statistical features have been presented for the CCPs recognition. In the feature selection module, the association rules (AR) method has been employed to select the best set of the shape and statistical features. In the classifier section, RBFNN is used and finally, in RBFNN, learning algorithm has a high impact on the network performance. Therefore, a new learning algorithm based on the bees algorithm has been used in the learning module. Most studies have considered only six patterns: Normal, Cyclic, Increasing Trend, Decreasing Trend, Upward Shift and Downward Shift. Since three patterns namely Normal, Stratification, and Systematic are very similar to each other and distinguishing them is very difficult, in most studies Stratification and Systematic have not been considered. Regarding to the continuous monitoring and control over the production process and the exact type detection of the problem encountered during the production process, eight patterns have been investigated in this study. The proposed method is tested on a dataset containing 1600 samples (200 samples from each pattern) and the results showed that the proposed method has a very good performance. Copyright © 2018 ISA. Published by Elsevier Ltd. All rights reserved.

Synergetic motor control paradigm for optimizing energy efficiency of multijoint reaching via tacit learning

PubMed Central

Hayashibe, Mitsuhiro; Shimoda, Shingo

2014-01-01

A human motor system can improve its behavior toward optimal movement. The skeletal system has more degrees of freedom than the task dimensions, which incurs an ill-posed problem. The multijoint system involves complex interaction torques between joints. To produce optimal motion in terms of energy consumption, the so-called cost function based optimization has been commonly used in previous works.Even if it is a fact that an optimal motor pattern is employed phenomenologically, there is no evidence that shows the existence of a physiological process that is similar to such a mathematical optimization in our central nervous system.In this study, we aim to find a more primitive computational mechanism with a modular configuration to realize adaptability and optimality without prior knowledge of system dynamics.We propose a novel motor control paradigm based on tacit learning with task space feedback. The motor command accumulation during repetitive environmental interactions, play a major role in the learning process. It is applied to a vertical cyclic reaching which involves complex interaction torques.We evaluated whether the proposed paradigm can learn how to optimize solutions with a 3-joint, planar biomechanical model. The results demonstrate that the proposed method was valid for acquiring motor synergy and resulted in energy efficient solutions for different load conditions. The case in feedback control is largely affected by the interaction torques. In contrast, the trajectory is corrected over time with tacit learning toward optimal solutions.Energy efficient solutions were obtained by the emergence of motor synergy. During learning, the contribution from feedforward controller is augmented and the one from the feedback controller is significantly minimized down to 12% for no load at hand, 16% for a 0.5 kg load condition.The proposed paradigm could provide an optimization process in redundant system with dynamic-model-free and cost-function-free approach. PMID:24616695

Synergetic motor control paradigm for optimizing energy efficiency of multijoint reaching via tacit learning.

PubMed

Hayashibe, Mitsuhiro; Shimoda, Shingo

2014-01-01

A human motor system can improve its behavior toward optimal movement. The skeletal system has more degrees of freedom than the task dimensions, which incurs an ill-posed problem. The multijoint system involves complex interaction torques between joints. To produce optimal motion in terms of energy consumption, the so-called cost function based optimization has been commonly used in previous works.Even if it is a fact that an optimal motor pattern is employed phenomenologically, there is no evidence that shows the existence of a physiological process that is similar to such a mathematical optimization in our central nervous system.In this study, we aim to find a more primitive computational mechanism with a modular configuration to realize adaptability and optimality without prior knowledge of system dynamics.We propose a novel motor control paradigm based on tacit learning with task space feedback. The motor command accumulation during repetitive environmental interactions, play a major role in the learning process. It is applied to a vertical cyclic reaching which involves complex interaction torques.We evaluated whether the proposed paradigm can learn how to optimize solutions with a 3-joint, planar biomechanical model. The results demonstrate that the proposed method was valid for acquiring motor synergy and resulted in energy efficient solutions for different load conditions. The case in feedback control is largely affected by the interaction torques. In contrast, the trajectory is corrected over time with tacit learning toward optimal solutions.Energy efficient solutions were obtained by the emergence of motor synergy. During learning, the contribution from feedforward controller is augmented and the one from the feedback controller is significantly minimized down to 12% for no load at hand, 16% for a 0.5 kg load condition.The proposed paradigm could provide an optimization process in redundant system with dynamic-model-free and cost-function-free approach.

Barrier Function-Based Neural Adaptive Control With Locally Weighted Learning and Finite Neuron Self-Growing Strategy.

PubMed

Jia, Zi-Jun; Song, Yong-Duan

2017-06-01

This paper presents a new approach to construct neural adaptive control for uncertain nonaffine systems. By integrating locally weighted learning with barrier Lyapunov function (BLF), a novel control design method is presented to systematically address the two critical issues in neural network (NN) control field: one is how to fulfill the compact set precondition for NN approximation, and the other is how to use varying rather than a fixed NN structure to improve the functionality of NN control. A BLF is exploited to ensure the NN inputs to remain bounded during the entire system operation. To account for system nonlinearities, a neuron self-growing strategy is proposed to guide the process for adding new neurons to the system, resulting in a self-adjustable NN structure for better learning capabilities. It is shown that the number of neurons needed to accomplish the control task is finite, and better performance can be obtained with less number of neurons as compared with traditional methods. The salient feature of the proposed method also lies in the continuity of the control action everywhere. Furthermore, the resulting control action is smooth almost everywhere except for a few time instants at which new neurons are added. Numerical example illustrates the effectiveness of the proposed approach.

Some problems with the design of self-learning management systems

NASA Technical Reports Server (NTRS)

Flikop, Ziny

1992-01-01

In this paper some problems in the design of management systems for complex objects are discussed. Considering the absence of adequate models and the fact that human expertise in the management of non-stationary objects becomes obsolete quickly, the use of self-learning together with a two-step optimization of on-line control rules is suggested. To prepare for the object analysis, a set of definitions has been proposed. Traditional and fuzzy sets approaches are used in the analysis. To decrease the reaction time of the control system, we propose the development of control rules without feedback.

Implementing Motivational Features in Reactive Blended Learning: Application to an Introductory Control Engineering Course

ERIC Educational Resources Information Center

Mendez, J. A.; Gonzalez, E. J.

2011-01-01

This paper presents a significant advance in a reactive blended learning methodology applied to an introductory control engineering course. This proposal was based on the inclusion of a reactive element (a fuzzy-logic-based controller) designed to regulate the workload for each student according to his/her activity and performance. The…

Models of Learning in ICAI.

ERIC Educational Resources Information Center

Duchastel, P.; And Others

1989-01-01

Discusses intelligent computer assisted instruction (ICAI) and presents various models of learning which have been proposed. Topics discussed include artificial intelligence; intelligent tutorial systems; tutorial strategies; learner control; system design; learning theory; and knowledge representation of proper and improper (i.e., incorrect)…

Tuning fuzzy PD and PI controllers using reinforcement learning.

PubMed

Boubertakh, Hamid; Tadjine, Mohamed; Glorennec, Pierre-Yves; Labiod, Salim

2010-10-01

In this paper, we propose a new auto-tuning fuzzy PD and PI controllers using reinforcement Q-learning (QL) algorithm for SISO (single-input single-output) and TITO (two-input two-output) systems. We first, investigate the design parameters and settings of a typical class of Fuzzy PD (FPD) and Fuzzy PI (FPI) controllers: zero-order Takagi-Sugeno controllers with equidistant triangular membership functions for inputs, equidistant singleton membership functions for output, Larsen's implication method, and average sum defuzzification method. Secondly, the analytical structures of these typical fuzzy PD and PI controllers are compared to their classical counterpart PD and PI controllers. Finally, the effectiveness of the proposed method is proven through simulation examples. Copyright © 2010 ISA. Published by Elsevier Ltd. All rights reserved.

Integral reinforcement learning for continuous-time input-affine nonlinear systems with simultaneous invariant explorations.

PubMed

Lee, Jae Young; Park, Jin Bae; Choi, Yoon Ho

2015-05-01

This paper focuses on a class of reinforcement learning (RL) algorithms, named integral RL (I-RL), that solve continuous-time (CT) nonlinear optimal control problems with input-affine system dynamics. First, we extend the concepts of exploration, integral temporal difference, and invariant admissibility to the target CT nonlinear system that is governed by a control policy plus a probing signal called an exploration. Then, we show input-to-state stability (ISS) and invariant admissibility of the closed-loop systems with the policies generated by integral policy iteration (I-PI) or invariantly admissible PI (IA-PI) method. Based on these, three online I-RL algorithms named explorized I-PI and integral Q -learning I, II are proposed, all of which generate the same convergent sequences as I-PI and IA-PI under the required excitation condition on the exploration. All the proposed methods are partially or completely model free, and can simultaneously explore the state space in a stable manner during the online learning processes. ISS, invariant admissibility, and convergence properties of the proposed methods are also investigated, and related with these, we show the design principles of the exploration for safe learning. Neural-network-based implementation methods for the proposed schemes are also presented in this paper. Finally, several numerical simulations are carried out to verify the effectiveness of the proposed methods.

Proposing a new iterative learning control algorithm based on a non-linear least square formulation - Minimising draw-in errors

NASA Astrophysics Data System (ADS)

Endelt, B.

2017-09-01

Forming operation are subject to external disturbances and changing operating conditions e.g. new material batch, increasing tool temperature due to plastic work, material properties and lubrication is sensitive to tool temperature. It is generally accepted that forming operations are not stable over time and it is not uncommon to adjust the process parameters during the first half hour production, indicating that process instability is gradually developing over time. Thus, in-process feedback control scheme might not-be necessary to stabilize the process and an alternative approach is to apply an iterative learning algorithm, which can learn from previously produced parts i.e. a self learning system which gradually reduces error based on historical process information. What is proposed in the paper is a simple algorithm which can be applied to a wide range of sheet-metal forming processes. The input to the algorithm is the final flange edge geometry and the basic idea is to reduce the least-square error between the current flange geometry and a reference geometry using a non-linear least square algorithm. The ILC scheme is applied to a square deep-drawing and the Numisheet’08 S-rail benchmark problem, the numerical tests shows that the proposed control scheme is able control and stabilise both processes.

GA-based fuzzy reinforcement learning for control of a magnetic bearing system.

PubMed

Lin, C T; Jou, C P

2000-01-01

This paper proposes a TD (temporal difference) and GA (genetic algorithm)-based reinforcement (TDGAR) learning method and applies it to the control of a real magnetic bearing system. The TDGAR learning scheme is a new hybrid GA, which integrates the TD prediction method and the GA to perform the reinforcement learning task. The TDGAR learning system is composed of two integrated feedforward networks. One neural network acts as a critic network to guide the learning of the other network (the action network) which determines the outputs (actions) of the TDGAR learning system. The action network can be a normal neural network or a neural fuzzy network. Using the TD prediction method, the critic network can predict the external reinforcement signal and provide a more informative internal reinforcement signal to the action network. The action network uses the GA to adapt itself according to the internal reinforcement signal. The key concept of the TDGAR learning scheme is to formulate the internal reinforcement signal as the fitness function for the GA such that the GA can evaluate the candidate solutions (chromosomes) regularly, even during periods without external feedback from the environment. This enables the GA to proceed to new generations regularly without waiting for the arrival of the external reinforcement signal. This can usually accelerate the GA learning since a reinforcement signal may only be available at a time long after a sequence of actions has occurred in the reinforcement learning problem. The proposed TDGAR learning system has been used to control an active magnetic bearing (AMB) system in practice. A systematic design procedure is developed to achieve successful integration of all the subsystems including magnetic suspension, mechanical structure, and controller training. The results show that the TDGAR learning scheme can successfully find a neural controller or a neural fuzzy controller for a self-designed magnetic bearing system.

The Seductive Waltz with the Self in Self-Regulated Learning: Toward Communal Regulation of Learning

ERIC Educational Resources Information Center

Schechter, Chen

2017-01-01

This article proposes a complementary framework for scholarship on metacognition as well as on self-regulated learning. It is argued that educators' and researchers' seductive waltz with the "self" in self-regulated learning (e.g., self-monitoring, self-control) need not be abandoned when conceptualizing and empirically investigating…

Coherent controlization using superconducting qubits

PubMed Central

Friis, Nicolai; Melnikov, Alexey A.; Kirchmair, Gerhard; Briegel, Hans J.

2015-01-01

Coherent controlization, i.e., coherent conditioning of arbitrary single- or multi-qubit operations on the state of one or more control qubits, is an important ingredient for the flexible implementation of many algorithms in quantum computation. This is of particular significance when certain subroutines are changing over time or when they are frequently modified, such as in decision-making algorithms for learning agents. We propose a scheme to realize coherent controlization for any number of superconducting qubits coupled to a microwave resonator. For two and three qubits, we present an explicit construction that is of high relevance for quantum learning agents. We demonstrate the feasibility of our proposal, taking into account loss, dephasing, and the cavity self-Kerr effect. PMID:26667893

Robust iterative learning control for multi-phase batch processes: an average dwell-time method with 2D convergence indexes

NASA Astrophysics Data System (ADS)

Wang, Limin; Shen, Yiteng; Yu, Jingxian; Li, Ping; Zhang, Ridong; Gao, Furong

2018-01-01

In order to cope with system disturbances in multi-phase batch processes with different dimensions, a hybrid robust control scheme of iterative learning control combined with feedback control is proposed in this paper. First, with a hybrid iterative learning control law designed by introducing the state error, the tracking error and the extended information, the multi-phase batch process is converted into a two-dimensional Fornasini-Marchesini (2D-FM) switched system with different dimensions. Second, a switching signal is designed using the average dwell-time method integrated with the related switching conditions to give sufficient conditions ensuring stable running for the system. Finally, the minimum running time of the subsystems and the control law gains are calculated by solving the linear matrix inequalities. Meanwhile, a compound 2D controller with robust performance is obtained, which includes a robust extended feedback control for ensuring the steady-state tracking error to converge rapidly. The application on an injection molding process displays the effectiveness and superiority of the proposed strategy.

Learning from ISS-modular adaptive NN control of nonlinear strict-feedback systems.

PubMed

Wang, Cong; Wang, Min; Liu, Tengfei; Hill, David J

2012-10-01

This paper studies learning from adaptive neural control (ANC) for a class of nonlinear strict-feedback systems with unknown affine terms. To achieve the purpose of learning, a simple input-to-state stability (ISS) modular ANC method is first presented to ensure the boundedness of all the signals in the closed-loop system and the convergence of tracking errors in finite time. Subsequently, it is proven that learning with the proposed stable ISS-modular ANC can be achieved. The cascade structure and unknown affine terms of the considered systems make it very difficult to achieve learning using existing methods. To overcome these difficulties, the stable closed-loop system in the control process is decomposed into a series of linear time-varying (LTV) perturbed subsystems with the appropriate state transformation. Using a recursive design, the partial persistent excitation condition for the radial basis function neural network (NN) is established, which guarantees exponential stability of LTV perturbed subsystems. Consequently, accurate approximation of the closed-loop system dynamics is achieved in a local region along recurrent orbits of closed-loop signals, and learning is implemented during a closed-loop feedback control process. The learned knowledge is reused to achieve stability and an improved performance, thereby avoiding the tremendous repeated training process of NNs. Simulation studies are given to demonstrate the effectiveness of the proposed method.

Self-learning fuzzy controllers based on temporal back propagation

NASA Technical Reports Server (NTRS)

Jang, Jyh-Shing R.

1992-01-01

This paper presents a generalized control strategy that enhances fuzzy controllers with self-learning capability for achieving prescribed control objectives in a near-optimal manner. This methodology, termed temporal back propagation, is model-insensitive in the sense that it can deal with plants that can be represented in a piecewise-differentiable format, such as difference equations, neural networks, GMDH structures, and fuzzy models. Regardless of the numbers of inputs and outputs of the plants under consideration, the proposed approach can either refine the fuzzy if-then rules if human experts, or automatically derive the fuzzy if-then rules obtained from human experts are not available. The inverted pendulum system is employed as a test-bed to demonstrate the effectiveness of the proposed control scheme and the robustness of the acquired fuzzy controller.

Adaptive Control Based Harvesting Strategy for a Predator-Prey Dynamical System.

PubMed

Sen, Moitri; Simha, Ashutosh; Raha, Soumyendu

2018-04-23

This paper deals with designing a harvesting control strategy for a predator-prey dynamical system, with parametric uncertainties and exogenous disturbances. A feedback control law for the harvesting rate of the predator is formulated such that the population dynamics is asymptotically stabilized at a positive operating point, while maintaining a positive, steady state harvesting rate. The hierarchical block strict feedback structure of the dynamics is exploited in designing a backstepping control law, based on Lyapunov theory. In order to account for unknown parameters, an adaptive control strategy has been proposed in which the control law depends on an adaptive variable which tracks the unknown parameter. Further, a switching component has been incorporated to robustify the control performance against bounded disturbances. Proofs have been provided to show that the proposed adaptive control strategy ensures asymptotic stability of the dynamics at a desired operating point, as well as exact parameter learning in the disturbance-free case and learning with bounded error in the disturbance prone case. The dynamics, with uncertainty in the death rate of the predator, subjected to a bounded disturbance has been simulated with the proposed control strategy.

Optimal control in microgrid using multi-agent reinforcement learning.

PubMed

Li, Fu-Dong; Wu, Min; He, Yong; Chen, Xin

2012-11-01

This paper presents an improved reinforcement learning method to minimize electricity costs on the premise of satisfying the power balance and generation limit of units in a microgrid with grid-connected mode. Firstly, the microgrid control requirements are analyzed and the objective function of optimal control for microgrid is proposed. Then, a state variable "Average Electricity Price Trend" which is used to express the most possible transitions of the system is developed so as to reduce the complexity and randomicity of the microgrid, and a multi-agent architecture including agents, state variables, action variables and reward function is formulated. Furthermore, dynamic hierarchical reinforcement learning, based on change rate of key state variable, is established to carry out optimal policy exploration. The analysis shows that the proposed method is beneficial to handle the problem of "curse of dimensionality" and speed up learning in the unknown large-scale world. Finally, the simulation results under JADE (Java Agent Development Framework) demonstrate the validity of the presented method in optimal control for a microgrid with grid-connected mode. Copyright © 2012 ISA. Published by Elsevier Ltd. All rights reserved.

Policy improvement by a model-free Dyna architecture.

PubMed

Hwang, Kao-Shing; Lo, Chia-Yue

2013-05-01

The objective of this paper is to accelerate the process of policy improvement in reinforcement learning. The proposed Dyna-style system combines two learning schemes, one of which utilizes a temporal difference method for direct learning; the other uses relative values for indirect learning in planning between two successive direct learning cycles. Instead of establishing a complicated world model, the approach introduces a simple predictor of average rewards to actor-critic architecture in the simulation (planning) mode. The relative value of a state, defined as the accumulated differences between immediate reward and average reward, is used to steer the improvement process in the right direction. The proposed learning scheme is applied to control a pendulum system for tracking a desired trajectory to demonstrate its adaptability and robustness. Through reinforcement signals from the environment, the system takes the appropriate action to drive an unknown dynamic to track desired outputs in few learning cycles. Comparisons are made between the proposed model-free method, a connectionist adaptive heuristic critic, and an advanced method of Dyna-Q learning in the experiments of labyrinth exploration. The proposed method outperforms its counterparts in terms of elapsed time and convergence rate.

Improving Grasp Skills Using Schema Structured Learning

NASA Technical Reports Server (NTRS)

Platt, Robert; Grupen, ROderic A.; Fagg, Andrew H.

2006-01-01

Abstract In the control-based approach to robotics, complex behavior is created by sequencing and combining control primitives. While it is desirable for the robot to autonomously learn the correct control sequence, searching through the large number of potential solutions can be time consuming. This paper constrains this search to variations of a generalized solution encoded in a framework known as an action schema. A new algorithm, SCHEMA STRUCTURED LEARNING, is proposed that repeatedly executes variations of the generalized solution in search of instantiations that satisfy action schema objectives. This approach is tested in a grasping task where Dexter, the UMass humanoid robot, learns which reaching and grasping controllers maximize the probability of grasp success.

Design of a completely model free adaptive control in the presence of parametric, non-parametric uncertainties and random control signal delay.

PubMed

Tutsoy, Onder; Barkana, Duygun Erol; Tugal, Harun

2018-05-01

In this paper, an adaptive controller is developed for discrete time linear systems that takes into account parametric uncertainty, internal-external non-parametric random uncertainties, and time varying control signal delay. Additionally, the proposed adaptive control is designed in such a way that it is utterly model free. Even though these properties are studied separately in the literature, they are not taken into account all together in adaptive control literature. The Q-function is used to estimate long-term performance of the proposed adaptive controller. Control policy is generated based on the long-term predicted value, and this policy searches an optimal stabilizing control signal for uncertain and unstable systems. The derived control law does not require an initial stabilizing control assumption as in the ones in the recent literature. Learning error, control signal convergence, minimized Q-function, and instantaneous reward are analyzed to demonstrate the stability and effectiveness of the proposed adaptive controller in a simulation environment. Finally, key insights on parameters convergence of the learning and control signals are provided. Copyright © 2018 ISA. Published by Elsevier Ltd. All rights reserved.

Lack of Interaction between Sensing-Intuitive Learning Styles and Problem-First versus Information-First Instruction: A Randomized Crossover Trial

ERIC Educational Resources Information Center

Cook, David A.; Thompson, Warren G.; Thomas, Kris G.; Thomas, Matthew R.

2009-01-01

Background: Adaptation to learning styles has been proposed to enhance learning. Objective: We hypothesized that learners with sensing learning style would perform better using a problem-first instructional method while intuitive learners would do better using an information-first method. Design: Randomized, controlled, crossover trial. Setting:…

Grounding cognitive control in associative learning.

PubMed

Abrahamse, Elger; Braem, Senne; Notebaert, Wim; Verguts, Tom

2016-07-01

Cognitive control covers a broad range of cognitive functions, but its research and theories typically remain tied to a single domain. Here we outline and review an associative learning perspective on cognitive control in which control emerges from associative networks containing perceptual, motor, and goal representations. Our review identifies 3 trending research themes that are shared between the domains of conflict adaptation, task switching, response inhibition, and attentional control: Cognitive control is context-specific, can operate in the absence of awareness, and is modulated by reward. As these research themes can be envisaged as key characteristics of learning, we propose that their joint emergence across domains is not coincidental but rather reflects a (latent) growth of interest in learning-based control. Associative learning has the potential for providing broad-scaled integration to cognitive control theory, and offers a promising avenue for understanding cognitive control as a self-regulating system without postulating an ill-defined set of homunculi. We discuss novel predictions, theoretical implications, and immediate challenges that accompany an associative learning perspective on cognitive control. (PsycINFO Database Record (c) 2016 APA, all rights reserved).

Action-Driven Visual Object Tracking With Deep Reinforcement Learning.

PubMed

Yun, Sangdoo; Choi, Jongwon; Yoo, Youngjoon; Yun, Kimin; Choi, Jin Young

2018-06-01

In this paper, we propose an efficient visual tracker, which directly captures a bounding box containing the target object in a video by means of sequential actions learned using deep neural networks. The proposed deep neural network to control tracking actions is pretrained using various training video sequences and fine-tuned during actual tracking for online adaptation to a change of target and background. The pretraining is done by utilizing deep reinforcement learning (RL) as well as supervised learning. The use of RL enables even partially labeled data to be successfully utilized for semisupervised learning. Through the evaluation of the object tracking benchmark data set, the proposed tracker is validated to achieve a competitive performance at three times the speed of existing deep network-based trackers. The fast version of the proposed method, which operates in real time on graphics processing unit, outperforms the state-of-the-art real-time trackers with an accuracy improvement of more than 8%.

Synchronization of Chaotic Systems without Direct Connections Using Reinforcement Learning

NASA Astrophysics Data System (ADS)

Sato, Norihisa; Adachi, Masaharu

In this paper, we propose a control method for the synchronization of chaotic systems that does not require the systems to be connected, unlike existing methods such as that proposed by Pecora and Carroll in 1990. The method is based on the reinforcement learning algorithm. We apply our method to two discrete-time chaotic systems with mismatched parameters and achieve M step delay synchronization. Moreover, we extend the proposed method to the synchronization of continuous-time chaotic systems.

Optimal critic learning for robot control in time-varying environments.

PubMed

Wang, Chen; Li, Yanan; Ge, Shuzhi Sam; Lee, Tong Heng

2015-10-01

In this paper, optimal critic learning is developed for robot control in a time-varying environment. The unknown environment is described as a linear system with time-varying parameters, and impedance control is employed for the interaction control. Desired impedance parameters are obtained in the sense of an optimal realization of the composite of trajectory tracking and force regulation. Q -function-based critic learning is developed to determine the optimal impedance parameters without the knowledge of the system dynamics. The simulation results are presented and compared with existing methods, and the efficacy of the proposed method is verified.

76 FR 76393 - Notice of Proposed Information Collection Requests

Federal Register 2010, 2011, 2012, 2013, 2014

2011-12-07

..., including through the use of information technology. Dated: December 2, 2011. Darrin King, Director... Collection: 21st Century Community Learning Centers: Lessons Learned Guides. OMB Control Number: 1875--NEW... Community Learning Centers (21st CCLC) program that will assist the U.S. Department of Education staff in...

Burnout and engagement at work as a function of demands and control.

PubMed

Demerouti, E; Bakker, A B; de Jonge, J; Janssen, P P; Schaufeli, W B

2001-08-01

The present study was designed to test the demand-control model using indicators of both health impairment and active learning or motivation. A total of 381 insurance company employees participated in the study. Discriminant analysis was used to examine the relationship between job demands and job control on one hand and health impairment and active learning on the other. The amount of demands and control could be predicted on the basis of employees' perceived health impairment (exhaustion and health complaints) and active learning (engagement and commitment). Each of the four combinations of demand and control differentially affected the perception of strain or active learning. Job demands were the most clearly related to health impairment, whereas job control was the most clearly associated with active learning. These findings partly contradict the demand-control model, especially with respect to the validity of the interaction between demand and control. Job demands and job control seem to initiate two essentially independent processes, and this occurrence is consistent with the recently proposed job demands-resources model.

HyFIS: adaptive neuro-fuzzy inference systems and their application to nonlinear dynamical systems.

PubMed

Kim, J; Kasabov, N

1999-11-01

This paper proposes an adaptive neuro-fuzzy system, HyFIS (Hybrid neural Fuzzy Inference System), for building and optimising fuzzy models. The proposed model introduces the learning power of neural networks to fuzzy logic systems and provides linguistic meaning to the connectionist architectures. Heuristic fuzzy logic rules and input-output fuzzy membership functions can be optimally tuned from training examples by a hybrid learning scheme comprised of two phases: rule generation phase from data; and rule tuning phase using error backpropagation learning scheme for a neural fuzzy system. To illustrate the performance and applicability of the proposed neuro-fuzzy hybrid model, extensive simulation studies of nonlinear complex dynamic systems are carried out. The proposed method can be applied to an on-line incremental adaptive learning for the prediction and control of nonlinear dynamical systems. Two benchmark case studies are used to demonstrate that the proposed HyFIS system is a superior neuro-fuzzy modelling technique.

Efficiency Improvement of Action Acquisition in Two-Link Robot Arm Using Fuzzy ART with Genetic Algorithm

NASA Astrophysics Data System (ADS)

Kotani, Naoki; Taniguchi, Kenji

An efficient learning method using Fuzzy ART with Genetic Algorithm is proposed. The proposed method reduces the number of trials by using a policy acquired in other tasks because a reinforcement learning needs a lot of the number of trials until an agent acquires appropriate actions. Fuzzy ART is an incremental unsupervised learning algorithm in responce to arbitrary sequences of analog or binary input vectors. Our proposed method gives a policy by crossover or mutation when an agent observes unknown states. Selection controls the category proliferation problem of Fuzzy ART. The effectiveness of the proposed method was verified with the simulation of the reaching problem for the two-link robot arm. The proposed method achieves a reduction of both the number of trials and the number of states.

A simple computational algorithm of model-based choice preference.

PubMed

Toyama, Asako; Katahira, Kentaro; Ohira, Hideki

2017-08-01

A broadly used computational framework posits that two learning systems operate in parallel during the learning of choice preferences-namely, the model-free and model-based reinforcement-learning systems. In this study, we examined another possibility, through which model-free learning is the basic system and model-based information is its modulator. Accordingly, we proposed several modified versions of a temporal-difference learning model to explain the choice-learning process. Using the two-stage decision task developed by Daw, Gershman, Seymour, Dayan, and Dolan (2011), we compared their original computational model, which assumes a parallel learning process, and our proposed models, which assume a sequential learning process. Choice data from 23 participants showed a better fit with the proposed models. More specifically, the proposed eligibility adjustment model, which assumes that the environmental model can weight the degree of the eligibility trace, can explain choices better under both model-free and model-based controls and has a simpler computational algorithm than the original model. In addition, the forgetting learning model and its variation, which assume changes in the values of unchosen actions, substantially improved the fits to the data. Overall, we show that a hybrid computational model best fits the data. The parameters used in this model succeed in capturing individual tendencies with respect to both model use in learning and exploration behavior. This computational model provides novel insights into learning with interacting model-free and model-based components.

Learner Control, User Characteristics, Platform Difference, and Their Role in Adoption Intention for MOOC Learning in China

ERIC Educational Resources Information Center

Zhang, Min; Yin, Shuaijun; Luo, Meifen; Yan, Weiwei

2017-01-01

Massive open online course (MOOC) learning attracts more and more attention in both the practice and the research field. Finding out what factors influence learners' MOOC adoption is of great importance. This study focuses on learner control, user characteristics and platform difference. Hypotheses and a research model are proposed by…

A Multiobjective Sparse Feature Learning Model for Deep Neural Networks.

PubMed

Gong, Maoguo; Liu, Jia; Li, Hao; Cai, Qing; Su, Linzhi

2015-12-01

Hierarchical deep neural networks are currently popular learning models for imitating the hierarchical architecture of human brain. Single-layer feature extractors are the bricks to build deep networks. Sparse feature learning models are popular models that can learn useful representations. But most of those models need a user-defined constant to control the sparsity of representations. In this paper, we propose a multiobjective sparse feature learning model based on the autoencoder. The parameters of the model are learnt by optimizing two objectives, reconstruction error and the sparsity of hidden units simultaneously to find a reasonable compromise between them automatically. We design a multiobjective induced learning procedure for this model based on a multiobjective evolutionary algorithm. In the experiments, we demonstrate that the learning procedure is effective, and the proposed multiobjective model can learn useful sparse features.

Approximately adaptive neural cooperative control for nonlinear multiagent systems with performance guarantee

NASA Astrophysics Data System (ADS)

Wang, Jing; Yang, Tianyu; Staskevich, Gennady; Abbe, Brian

2017-04-01

This paper studies the cooperative control problem for a class of multiagent dynamical systems with partially unknown nonlinear system dynamics. In particular, the control objective is to solve the state consensus problem for multiagent systems based on the minimisation of certain cost functions for individual agents. Under the assumption that there exist admissible cooperative controls for such class of multiagent systems, the formulated problem is solved through finding the optimal cooperative control using the approximate dynamic programming and reinforcement learning approach. With the aid of neural network parameterisation and online adaptive learning, our method renders a practically implementable approximately adaptive neural cooperative control for multiagent systems. Specifically, based on the Bellman's principle of optimality, the Hamilton-Jacobi-Bellman (HJB) equation for multiagent systems is first derived. We then propose an approximately adaptive policy iteration algorithm for multiagent cooperative control based on neural network approximation of the value functions. The convergence of the proposed algorithm is rigorously proved using the contraction mapping method. The simulation results are included to validate the effectiveness of the proposed algorithm.

A Policy Representation Using Weighted Multiple Normal Distribution

NASA Astrophysics Data System (ADS)

Kimura, Hajime; Aramaki, Takeshi; Kobayashi, Shigenobu

In this paper, we challenge to solve a reinforcement learning problem for a 5-linked ring robot within a real-time so that the real-robot can stand up to the trial and error. On this robot, incomplete perception problems are caused from noisy sensors and cheap position-control motor systems. This incomplete perception also causes varying optimum actions with the progress of the learning. To cope with this problem, we adopt an actor-critic method, and we propose a new hierarchical policy representation scheme, that consists of discrete action selection on the top level and continuous action selection on the low level of the hierarchy. The proposed hierarchical scheme accelerates learning on continuous action space, and it can pursue the optimum actions varying with the progress of learning on our robotics problem. This paper compares and discusses several learning algorithms through simulations, and demonstrates the proposed method showing application for the real robot.

Design issues for a reinforcement-based self-learning fuzzy controller

NASA Technical Reports Server (NTRS)

Yen, John; Wang, Haojin; Dauherity, Walter

1993-01-01

Fuzzy logic controllers have some often cited advantages over conventional techniques such as PID control: easy implementation, its accommodation to natural language, the ability to cover wider range of operating conditions and others. One major obstacle that hinders its broader application is the lack of a systematic way to develop and modify its rules and as result the creation and modification of fuzzy rules often depends on try-error or pure experimentation. One of the proposed approaches to address this issue is self-learning fuzzy logic controllers (SFLC) that use reinforcement learning techniques to learn the desirability of states and to adjust the consequent part of fuzzy control rules accordingly. Due to the different dynamics of the controlled processes, the performance of self-learning fuzzy controller is highly contingent on the design. The design issue has not received sufficient attention. The issues related to the design of a SFLC for the application to chemical process are discussed and its performance is compared with that of PID and self-tuning fuzzy logic controller.

Systematic construction and control of stereo nerve vision network in intelligent manufacturing

NASA Astrophysics Data System (ADS)

Liu, Hua; Wang, Helong; Guo, Chunjie; Ding, Quanxin; Zhou, Liwei

2017-10-01

A system method of constructing stereo vision by using neural network is proposed, and the operation and control mechanism in actual operation are proposed. This method makes effective use of the neural network in learning and memory function, by after training with samples. Moreover, the neural network can learn the nonlinear relationship in the stereoscopic vision system and the internal and external orientation elements. These considerations are Worthy of attention, which includes limited constraints, the scientific of critical group, the operating speed and the operability in technical aspects. The results support our theoretical forecast.

Joint Machine Learning and Game Theory for Rate Control in High Efficiency Video Coding.

PubMed

Gao, Wei; Kwong, Sam; Jia, Yuheng

2017-08-25

In this paper, a joint machine learning and game theory modeling (MLGT) framework is proposed for inter frame coding tree unit (CTU) level bit allocation and rate control (RC) optimization in High Efficiency Video Coding (HEVC). First, a support vector machine (SVM) based multi-classification scheme is proposed to improve the prediction accuracy of CTU-level Rate-Distortion (R-D) model. The legacy "chicken-and-egg" dilemma in video coding is proposed to be overcome by the learning-based R-D model. Second, a mixed R-D model based cooperative bargaining game theory is proposed for bit allocation optimization, where the convexity of the mixed R-D model based utility function is proved, and Nash bargaining solution (NBS) is achieved by the proposed iterative solution search method. The minimum utility is adjusted by the reference coding distortion and frame-level Quantization parameter (QP) change. Lastly, intra frame QP and inter frame adaptive bit ratios are adjusted to make inter frames have more bit resources to maintain smooth quality and bit consumption in the bargaining game optimization. Experimental results demonstrate that the proposed MLGT based RC method can achieve much better R-D performances, quality smoothness, bit rate accuracy, buffer control results and subjective visual quality than the other state-of-the-art one-pass RC methods, and the achieved R-D performances are very close to the performance limits from the FixedQP method.

Autonomous reinforcement learning with experience replay.

PubMed

Wawrzyński, Paweł; Tanwani, Ajay Kumar

2013-05-01

This paper considers the issues of efficiency and autonomy that are required to make reinforcement learning suitable for real-life control tasks. A real-time reinforcement learning algorithm is presented that repeatedly adjusts the control policy with the use of previously collected samples, and autonomously estimates the appropriate step-sizes for the learning updates. The algorithm is based on the actor-critic with experience replay whose step-sizes are determined on-line by an enhanced fixed point algorithm for on-line neural network training. An experimental study with simulated octopus arm and half-cheetah demonstrates the feasibility of the proposed algorithm to solve difficult learning control problems in an autonomous way within reasonably short time. Copyright © 2012 Elsevier Ltd. All rights reserved.

Fault-tolerant optimised tracking control for unknown discrete-time linear systems using a combined reinforcement learning and residual compensation methodology

NASA Astrophysics Data System (ADS)

Han, Ke-Zhen; Feng, Jian; Cui, Xiaohong

2017-10-01

This paper considers the fault-tolerant optimised tracking control (FTOTC) problem for unknown discrete-time linear system. A research scheme is proposed on the basis of data-based parity space identification, reinforcement learning and residual compensation techniques. The main characteristic of this research scheme lies in the parity-space-identification-based simultaneous tracking control and residual compensation. The specific technical line consists of four main contents: apply subspace aided method to design observer-based residual generator; use reinforcement Q-learning approach to solve optimised tracking control policy; rely on robust H∞ theory to achieve noise attenuation; adopt fault estimation triggered by residual generator to perform fault compensation. To clarify the design and implementation procedures, an integrated algorithm is further constructed to link up these four functional units. The detailed analysis and proof are subsequently given to explain the guaranteed FTOTC performance of the proposed conclusions. Finally, a case simulation is provided to verify its effectiveness.

Peer Assisted Learning in the Clinical Setting: An Activity Systems Analysis

ERIC Educational Resources Information Center

Bennett, Deirdre; O'Flynn, Siun; Kelly, Martina

2015-01-01

Peer assisted learning (PAL) is a common feature of medical education. Understanding of PAL has been based on processes and outcomes in controlled settings, such as clinical skills labs. PAL in the clinical setting, a complex learning environment, requires fresh evaluation. Socio-cultural theory is proposed as a means to understand educational…

Simulating Conditions of Learned Helplessness: The Effects of Interventions and Attributions.

ERIC Educational Resources Information Center

Donovan, Wilberta L.; Leavitt, Lewis A.

1985-01-01

Using a version of the "learned helplessness" paradigm, assesses mothers' performance on a solvable task following pretreatments that involved exposure to an infant cry but that differed in the mothers' ability to exert control over termination of the cry. Proposes that learned helplessness models are relevant to the study of…

Organizational Socialization: A Social Learning Interpretation

DTIC Science & Technology

1982-02-01

approaches to socialization, they lack a clear theoretical basis for understanding and application. This paper proposes a social learning theoretical ... framework . Particular attention is given to the relevancy that modeling and self-control can have for organizational socialization. Specific examples of

Bio-inspired adaptive feedback error learning architecture for motor control.

PubMed

Tolu, Silvia; Vanegas, Mauricio; Luque, Niceto R; Garrido, Jesús A; Ros, Eduardo

2012-10-01

This study proposes an adaptive control architecture based on an accurate regression method called Locally Weighted Projection Regression (LWPR) and on a bio-inspired module, such as a cerebellar-like engine. This hybrid architecture takes full advantage of the machine learning module (LWPR kernel) to abstract an optimized representation of the sensorimotor space while the cerebellar component integrates this to generate corrective terms in the framework of a control task. Furthermore, we illustrate how the use of a simple adaptive error feedback term allows to use the proposed architecture even in the absence of an accurate analytic reference model. The presented approach achieves an accurate control with low gain corrective terms (for compliant control schemes). We evaluate the contribution of the different components of the proposed scheme comparing the obtained performance with alternative approaches. Then, we show that the presented architecture can be used for accurate manipulation of different objects when their physical properties are not directly known by the controller. We evaluate how the scheme scales for simulated plants of high Degrees of Freedom (7-DOFs).

Model-Free Adaptive Control for Unknown Nonlinear Zero-Sum Differential Game.

PubMed

Zhong, Xiangnan; He, Haibo; Wang, Ding; Ni, Zhen

2018-05-01

In this paper, we present a new model-free globalized dual heuristic dynamic programming (GDHP) approach for the discrete-time nonlinear zero-sum game problems. First, the online learning algorithm is proposed based on the GDHP method to solve the Hamilton-Jacobi-Isaacs equation associated with optimal regulation control problem. By setting backward one step of the definition of performance index, the requirement of system dynamics, or an identifier is relaxed in the proposed method. Then, three neural networks are established to approximate the optimal saddle point feedback control law, the disturbance law, and the performance index, respectively. The explicit updating rules for these three neural networks are provided based on the data generated during the online learning along the system trajectories. The stability analysis in terms of the neural network approximation errors is discussed based on the Lyapunov approach. Finally, two simulation examples are provided to show the effectiveness of the proposed method.

Learning-Based Adaptive Optimal Tracking Control of Strict-Feedback Nonlinear Systems.

PubMed

Gao, Weinan; Jiang, Zhong-Ping; Weinan Gao; Zhong-Ping Jiang; Gao, Weinan; Jiang, Zhong-Ping

2018-06-01

This paper proposes a novel data-driven control approach to address the problem of adaptive optimal tracking for a class of nonlinear systems taking the strict-feedback form. Adaptive dynamic programming (ADP) and nonlinear output regulation theories are integrated for the first time to compute an adaptive near-optimal tracker without any a priori knowledge of the system dynamics. Fundamentally different from adaptive optimal stabilization problems, the solution to a Hamilton-Jacobi-Bellman (HJB) equation, not necessarily a positive definite function, cannot be approximated through the existing iterative methods. This paper proposes a novel policy iteration technique for solving positive semidefinite HJB equations with rigorous convergence analysis. A two-phase data-driven learning method is developed and implemented online by ADP. The efficacy of the proposed adaptive optimal tracking control methodology is demonstrated via a Van der Pol oscillator with time-varying exogenous signals.

Adaptive critic learning techniques for engine torque and air-fuel ratio control.

PubMed

Liu, Derong; Javaherian, Hossein; Kovalenko, Olesia; Huang, Ting

2008-08-01

A new approach for engine calibration and control is proposed. In this paper, we present our research results on the implementation of adaptive critic designs for self-learning control of automotive engines. A class of adaptive critic designs that can be classified as (model-free) action-dependent heuristic dynamic programming is used in this research project. The goals of the present learning control design for automotive engines include improved performance, reduced emissions, and maintained optimum performance under various operating conditions. Using the data from a test vehicle with a V8 engine, we developed a neural network model of the engine and neural network controllers based on the idea of approximate dynamic programming to achieve optimal control. We have developed and simulated self-learning neural network controllers for both engine torque (TRQ) and exhaust air-fuel ratio (AFR) control. The goal of TRQ control and AFR control is to track the commanded values. For both control problems, excellent neural network controller transient performance has been achieved.

Long-term effects of controllability or the lack of it on coping abilities and stress resilience in the rat.

PubMed

Lucas, Morgan; Ilin, Yana; Anunu, Rachel; Kehat, Orli; Xu, Lin; Desmedt, Aline; Richter-Levin, Gal

2014-09-01

Findings suggest that stress-induced impaired learning and coping abilities may be attributed more to the psychological nature of the stressor, rather than its physical properties. It has been proposed that establishing controllability over stressors can ameliorate some of its effects on cognition and behavior. Gaining controllability was suggested to be associated with the development of stress resilience. Based on repeated exposure to the two-way shuttle avoidance task, we previously developed and validated a behavioral task that leads to a strict dissociation between gaining controllability (to the level that the associated fear is significantly reduced) and a fearful state of uncontrollability. Employing this protocol, we investigated here the impact of gaining or failing to gain emotional controllability on indices of anxiety and depression and on subsequent abilities to cope with positively or negatively reinforcing learning experiences. In agreement with previous studies, rats exposed to the uncontrollable protocol demonstrated high concentration of sera corticosterone, increased immobility, reduced duration of struggling in the forced swim test and impaired ability to acquire subsequent learning tasks. Achieving emotional controllability resulted in resilience to stress as was indicated by longer duration of struggling in the forced swim test, and enhanced learning abilities. Our prolonged training protocol, with the demonstrated ability of rats to gain emotional controllability, is proposed as a useful tool to study the neurobiological mechanisms of stress resilience.

Design issues of a reinforcement-based self-learning fuzzy controller for petrochemical process control

NASA Technical Reports Server (NTRS)

Yen, John; Wang, Haojin; Daugherity, Walter C.

1992-01-01

Fuzzy logic controllers have some often-cited advantages over conventional techniques such as PID control, including easier implementation, accommodation to natural language, and the ability to cover a wider range of operating conditions. One major obstacle that hinders the broader application of fuzzy logic controllers is the lack of a systematic way to develop and modify their rules; as a result the creation and modification of fuzzy rules often depends on trial and error or pure experimentation. One of the proposed approaches to address this issue is a self-learning fuzzy logic controller (SFLC) that uses reinforcement learning techniques to learn the desirability of states and to adjust the consequent part of its fuzzy control rules accordingly. Due to the different dynamics of the controlled processes, the performance of a self-learning fuzzy controller is highly contingent on its design. The design issue has not received sufficient attention. The issues related to the design of a SFLC for application to a petrochemical process are discussed, and its performance is compared with that of a PID and a self-tuning fuzzy logic controller.

A Telescopic Binary Learning Machine for Training Neural Networks.

PubMed

Brunato, Mauro; Battiti, Roberto

2017-03-01

This paper proposes a new algorithm based on multiscale stochastic local search with binary representation for training neural networks [binary learning machine (BLM)]. We study the effects of neighborhood evaluation strategies, the effect of the number of bits per weight and that of the maximum weight range used for mapping binary strings to real values. Following this preliminary investigation, we propose a telescopic multiscale version of local search, where the number of bits is increased in an adaptive manner, leading to a faster search and to local minima of better quality. An analysis related to adapting the number of bits in a dynamic way is presented. The control on the number of bits, which happens in a natural manner in the proposed method, is effective to increase the generalization performance. The learning dynamics are discussed and validated on a highly nonlinear artificial problem and on real-world tasks in many application domains; BLM is finally applied to a problem requiring either feedforward or recurrent architectures for feedback control.

Gr-GDHP: A New Architecture for Globalized Dual Heuristic Dynamic Programming.

PubMed

Zhong, Xiangnan; Ni, Zhen; He, Haibo

2017-10-01

Goal representation globalized dual heuristic dynamic programming (Gr-GDHP) method is proposed in this paper. A goal neural network is integrated into the traditional GDHP method providing an internal reinforcement signal and its derivatives to help the control and learning process. From the proposed architecture, it is shown that the obtained internal reinforcement signal and its derivatives can be able to adjust themselves online over time rather than a fixed or predefined function in literature. Furthermore, the obtained derivatives can directly contribute to the objective function of the critic network, whose learning process is thus simplified. Numerical simulation studies are applied to show the performance of the proposed Gr-GDHP method and compare the results with other existing adaptive dynamic programming designs. We also investigate this method on a ball-and-beam balancing system. The statistical simulation results are presented for both the Gr-GDHP and the GDHP methods to demonstrate the improved learning and controlling performance.

Robust iterative learning contouring controller with disturbance observer for machine tool feed drives.

PubMed

Simba, Kenneth Renny; Bui, Ba Dinh; Msukwa, Mathew Renny; Uchiyama, Naoki

2018-04-01

In feed drive systems, particularly machine tools, a contour error is more significant than the individual axial tracking errors from the view point of enhancing precision in manufacturing and production systems. The contour error must be within the permissible tolerance of given products. In machining complex or sharp-corner products, large contour errors occur mainly owing to discontinuous trajectories and the existence of nonlinear uncertainties. Therefore, it is indispensable to design robust controllers that can enhance the tracking ability of feed drive systems. In this study, an iterative learning contouring controller consisting of a classical Proportional-Derivative (PD) controller and disturbance observer is proposed. The proposed controller was evaluated experimentally by using a typical sharp-corner trajectory, and its performance was compared with that of conventional controllers. The results revealed that the maximum contour error can be reduced by about 37% on average. Copyright © 2018 ISA. Published by Elsevier Ltd. All rights reserved.

Tracked robot controllers for climbing obstacles autonomously

NASA Astrophysics Data System (ADS)

Vincent, Isabelle

2009-05-01

Research in mobile robot navigation has demonstrated some success in navigating flat indoor environments while avoiding obstacles. However, the challenge of analyzing complex environments to climb obstacles autonomously has had very little success due to the complexity of the task. Unmanned ground vehicles currently exhibit simple autonomous behaviours compared to the human ability to move in the world. This paper presents the control algorithms designed for a tracked mobile robot to autonomously climb obstacles by varying its tracks configuration. Two control algorithms are proposed to solve the autonomous locomotion problem for climbing obstacles. First, a reactive controller evaluates the appropriate geometric configuration based on terrain and vehicle geometric considerations. Then, a reinforcement learning algorithm finds alternative solutions when the reactive controller gets stuck while climbing an obstacle. The methodology combines reactivity to learning. The controllers have been demonstrated in box and stair climbing simulations. The experiments illustrate the effectiveness of the proposed approach for crossing obstacles.

New education system for construction of optical holography setup - Tangible learning with Augmented Reality

NASA Astrophysics Data System (ADS)

Yamaguchi, Takeshi; Yoshikawa, Hiroshi

2013-02-01

In case of teaching optical system construction, it is difficult to prepare the optical components for the attendance student. However the tangible learning is very important to master the optical system construction. It helps learners understand easily to use an inexpensive learning system that provides optical experiments experiences. Therefore, we propose the new education system for construction of optical setup with the augmented reality. To use the augmented reality, the proposed system can simulate the optical system construction by the direct hand control. Also, this system only requires an inexpensive web camera, printed makers and a personal computer. Since this system does not require the darkroom and the expensive optical equipments, the learners can study anytime, anywhere when they want to do. In this paper, we developed the system that can teach the optical system construction of the Denisyuk hologram and 2-step transmission type hologram. For the tangible learning and the easy understanding, the proposed system displays the CG objects of the optical components on the markers which are controlled by the learner's hands. The proposed system does not only display the CG object, but also display the light beam which is controlled by the optical components. To display the light beam that is hard to be seen directly, the learners can confirm about what is happening by the own manipulation. For the construction of optical holography setup, we arrange a laser, mirrors, a PBS (polarizing beam splitter), lenses, a polarizer, half-wave plates, spatial filters, an optical power meter and a recording plate. After the construction, proposed system can check optical setup correctly. In comparison with the learners who only read a book, the learners who use the system can construct the optical holography setup more quickly and correctly.

Spiral and Project-Based Learning with Peer Assessment in a Computer Science Project Management Course

NASA Astrophysics Data System (ADS)

Jaime, Arturo; Blanco, José Miguel; Domínguez, César; Sánchez, Ana; Heras, Jónathan; Usandizaga, Imanol

2016-06-01

Different learning methods such as project-based learning, spiral learning and peer assessment have been implemented in science disciplines with different outcomes. This paper presents a proposal for a project management course in the context of a computer science degree. Our proposal combines three well-known methods: project-based learning, spiral learning and peer assessment. Namely, the course is articulated during a semester through the structured (progressive and incremental) development of a sequence of four projects, whose duration, scope and difficulty of management increase as the student gains theoretical and instrumental knowledge related to planning, monitoring and controlling projects. Moreover, the proposal is complemented using peer assessment. The proposal has already been implemented and validated for the last 3 years in two different universities. In the first year, project-based learning and spiral learning methods were combined. Such a combination was also employed in the other 2 years; but additionally, students had the opportunity to assess projects developed by university partners and by students of the other university. A total of 154 students have participated in the study. We obtain a gain in the quality of the subsequently projects derived from the spiral project-based learning. Moreover, this gain is significantly bigger when peer assessment is introduced. In addition, high-performance students take advantage of peer assessment from the first moment, whereas the improvement in poor-performance students is delayed.

Reinforcement learning techniques for controlling resources in power networks

NASA Astrophysics Data System (ADS)

Kowli, Anupama Sunil

As power grids transition towards increased reliance on renewable generation, energy storage and demand response resources, an effective control architecture is required to harness the full functionalities of these resources. There is a critical need for control techniques that recognize the unique characteristics of the different resources and exploit the flexibility afforded by them to provide ancillary services to the grid. The work presented in this dissertation addresses these needs. Specifically, new algorithms are proposed, which allow control synthesis in settings wherein the precise distribution of the uncertainty and its temporal statistics are not known. These algorithms are based on recent developments in Markov decision theory, approximate dynamic programming and reinforcement learning. They impose minimal assumptions on the system model and allow the control to be "learned" based on the actual dynamics of the system. Furthermore, they can accommodate complex constraints such as capacity and ramping limits on generation resources, state-of-charge constraints on storage resources, comfort-related limitations on demand response resources and power flow limits on transmission lines. Numerical studies demonstrating applications of these algorithms to practical control problems in power systems are discussed. Results demonstrate how the proposed control algorithms can be used to improve the performance and reduce the computational complexity of the economic dispatch mechanism in a power network. We argue that the proposed algorithms are eminently suitable to develop operational decision-making tools for large power grids with many resources and many sources of uncertainty.

Efficient model learning methods for actor-critic control.

PubMed

Grondman, Ivo; Vaandrager, Maarten; Buşoniu, Lucian; Babuska, Robert; Schuitema, Erik

2012-06-01

We propose two new actor-critic algorithms for reinforcement learning. Both algorithms use local linear regression (LLR) to learn approximations of the functions involved. A crucial feature of the algorithms is that they also learn a process model, and this, in combination with LLR, provides an efficient policy update for faster learning. The first algorithm uses a novel model-based update rule for the actor parameters. The second algorithm does not use an explicit actor but learns a reference model which represents a desired behavior, from which desired control actions can be calculated using the inverse of the learned process model. The two novel methods and a standard actor-critic algorithm are applied to the pendulum swing-up problem, in which the novel methods achieve faster learning than the standard algorithm.

Cooperative learning neural network output feedback control of uncertain nonlinear multi-agent systems under directed topologies

NASA Astrophysics Data System (ADS)

Wang, W.; Wang, D.; Peng, Z. H.

2017-09-01

Without assuming that the communication topologies among the neural network (NN) weights are to be undirected and the states of each agent are measurable, the cooperative learning NN output feedback control is addressed for uncertain nonlinear multi-agent systems with identical structures in strict-feedback form. By establishing directed communication topologies among NN weights to share their learned knowledge, NNs with cooperative learning laws are employed to identify the uncertainties. By designing NN-based κ-filter observers to estimate the unmeasurable states, a new cooperative learning output feedback control scheme is proposed to guarantee that the system outputs can track nonidentical reference signals with bounded tracking errors. A simulation example is given to demonstrate the effectiveness of the theoretical results.

Toward a Model for the Conceptual Understanding of Personal Learning Environments: A Case Study

ERIC Educational Resources Information Center

Ivanova, Malinka; Chatti, Mohamed Amine

2011-01-01

The development of Personal Learning Environments (PLEs) is in the scope of research groups and educators aiming to propose suitable mechanisms for the organization of self-controlled and self-directed learning, providing students with tools and services for access to content and human intelligence inside and outside the educational institutions.…

Fuzzy adaptive iterative learning coordination control of second-order multi-agent systems with imprecise communication topology structure

NASA Astrophysics Data System (ADS)

Chen, Jiaxi; Li, Junmin

2018-02-01

In this paper, we investigate the perfect consensus problem for second-order linearly parameterised multi-agent systems (MAS) with imprecise communication topology structure. Takagi-Sugeno (T-S) fuzzy models are presented to describe the imprecise communication topology structure of leader-following MAS, and a distributed adaptive iterative learning control protocol is proposed with the dynamic of leader unknown to any of the agent. The proposed protocol guarantees that the follower agents can track the leader perfectly on [0,T] for the consensus problem. Under alignment condition, a sufficient condition of the consensus for closed-loop MAS is given based on Lyapunov stability theory. Finally, a numerical example and a multiple pendulum system are given to illustrate the effectiveness of the proposed algorithm.

A proposal of an architecture for the coordination level of intelligent machines

NASA Technical Reports Server (NTRS)

Beard, Randall; Farah, Jeff; Lima, Pedro

1993-01-01

The issue of obtaining a practical, structured, and detailed description of an architecture for the Coordination Level of Center for Intelligent Robotic Systems for Sapce Exploration (CIRSSE) Testbed Intelligent Controller is addressed. Previous theoretical and implementation works were the departure point for the discussion. The document is organized as follows: after this introductory section, section 2 summarizes the overall view of the Intelligent Machine (IM) as a control system, proposing a performance measure on which to base its design. Section 3 addresses with some detail implementation issues. An hierarchic petri-net with feedback-based learning capabilities is proposed. Finally, section 4 is an attempt to address the feedback problem. Feedback is used for two functions: error recovery and reinforcement learning of the correct translations for the petri-net transitions.

Weakly Supervised Dictionary Learning

NASA Astrophysics Data System (ADS)

You, Zeyu; Raich, Raviv; Fern, Xiaoli Z.; Kim, Jinsub

2018-05-01

We present a probabilistic modeling and inference framework for discriminative analysis dictionary learning under a weak supervision setting. Dictionary learning approaches have been widely used for tasks such as low-level signal denoising and restoration as well as high-level classification tasks, which can be applied to audio and image analysis. Synthesis dictionary learning aims at jointly learning a dictionary and corresponding sparse coefficients to provide accurate data representation. This approach is useful for denoising and signal restoration, but may lead to sub-optimal classification performance. By contrast, analysis dictionary learning provides a transform that maps data to a sparse discriminative representation suitable for classification. We consider the problem of analysis dictionary learning for time-series data under a weak supervision setting in which signals are assigned with a global label instead of an instantaneous label signal. We propose a discriminative probabilistic model that incorporates both label information and sparsity constraints on the underlying latent instantaneous label signal using cardinality control. We present the expectation maximization (EM) procedure for maximum likelihood estimation (MLE) of the proposed model. To facilitate a computationally efficient E-step, we propose both a chain and a novel tree graph reformulation of the graphical model. The performance of the proposed model is demonstrated on both synthetic and real-world data.

An approach for investigation of secure access processes at a combined e-learning environment

NASA Astrophysics Data System (ADS)

Romansky, Radi; Noninska, Irina

2017-12-01

The article discuses an approach to investigate processes for regulation the security and privacy control at a heterogenous e-learning environment realized as a combination of traditional and cloud means and tools. Authors' proposal for combined architecture of e-learning system is presented and main subsystems and procedures are discussed. A formalization of the processes for using different types resources (public, private internal and private external) is proposed. The apparatus of Markovian chains (MC) is used for modeling and analytical investigation of the secure access to the resources is used and some assessments are presented.

Building environment analysis based on temperature and humidity for smart energy systems.

PubMed

Yun, Jaeseok; Won, Kwang-Ho

2012-10-01

In this paper, we propose a new HVAC (heating, ventilation, and air conditioning) control strategy as part of the smart energy system that can balance occupant comfort against building energy consumption using ubiquitous sensing and machine learning technology. We have developed ZigBee-based wireless sensor nodes and collected realistic temperature and humidity data during one month from a laboratory environment. With the collected data, we have established a building environment model using machine learning algorithms, which can be used to assess occupant comfort level. We expect the proposed HVAC control strategy will be able to provide occupants with a consistently comfortable working or home environment.

A new computational account of cognitive control over reinforcement-based decision-making: Modeling of a probabilistic learning task.

PubMed

Zendehrouh, Sareh

2015-11-01

Recent work on decision-making field offers an account of dual-system theory for decision-making process. This theory holds that this process is conducted by two main controllers: a goal-directed system and a habitual system. In the reinforcement learning (RL) domain, the habitual behaviors are connected with model-free methods, in which appropriate actions are learned through trial-and-error experiences. However, goal-directed behaviors are associated with model-based methods of RL, in which actions are selected using a model of the environment. Studies on cognitive control also suggest that during processes like decision-making, some cortical and subcortical structures work in concert to monitor the consequences of decisions and to adjust control according to current task demands. Here a computational model is presented based on dual system theory and cognitive control perspective of decision-making. The proposed model is used to simulate human performance on a variant of probabilistic learning task. The basic proposal is that the brain implements a dual controller, while an accompanying monitoring system detects some kinds of conflict including a hypothetical cost-conflict one. The simulation results address existing theories about two event-related potentials, namely error related negativity (ERN) and feedback related negativity (FRN), and explore the best account of them. Based on the results, some testable predictions are also presented. Copyright © 2015 Elsevier Ltd. All rights reserved.

Flyback CCM inverter for AC module applications: iterative learning control and convergence analysis

NASA Astrophysics Data System (ADS)

Lee, Sung-Ho; Kim, Minsung

2017-12-01

This paper presents an iterative learning controller (ILC) for an interleaved flyback inverter operating in continuous conduction mode (CCM). The flyback CCM inverter features small output ripple current, high efficiency, and low cost, and hence it is well suited for photovoltaic power applications. However, it exhibits the non-minimum phase behaviour, because its transfer function from control duty to output current has the right-half-plane (RHP) zero. Moreover, the flyback CCM inverter suffers from the time-varying grid voltage disturbance. Thus, conventional control scheme results in inaccurate output tracking. To overcome these problems, the ILC is first developed and applied to the flyback inverter operating in CCM. The ILC makes use of both predictive and current learning terms which help the system output to converge to the reference trajectory. We take into account the nonlinear averaged model and use it to construct the proposed controller. It is proven that the system output globally converges to the reference trajectory in the absence of state disturbances, output noises, or initial state errors. Numerical simulations are performed to validate the proposed control scheme, and experiments using 400-W AC module prototype are carried out to demonstrate its practical feasibility.

Decentralized reinforcement-learning control and emergence of motion patterns

NASA Astrophysics Data System (ADS)

Svinin, Mikhail; Yamada, Kazuyaki; Okhura, Kazuhiro; Ueda, Kanji

1998-10-01

In this paper we propose a system for studying emergence of motion patterns in autonomous mobile robotic systems. The system implements an instance-based reinforcement learning control. Three spaces are of importance in formulation of the control scheme. They are the work space, the sensor space, and the action space. Important feature of our system is that all these spaces are assumed to be continuous. The core part of the system is a classifier system. Based on the sensory state space analysis, the control is decentralized and is specified at the lowest level of the control system. However, the local controllers are implicitly connected through the perceived environment information. Therefore, they constitute a dynamic environment with respect to each other. The proposed control scheme is tested under simulation for a mobile robot in a navigation task. It is shown that some patterns of global behavior--such as collision avoidance, wall-following, light-seeking--can emerge from the local controllers.

Neural network-based model reference adaptive control system.

PubMed

Patino, H D; Liu, D

2000-01-01

In this paper, an approach to model reference adaptive control based on neural networks is proposed and analyzed for a class of first-order continuous-time nonlinear dynamical systems. The controller structure can employ either a radial basis function network or a feedforward neural network to compensate adaptively the nonlinearities in the plant. A stable controller-parameter adjustment mechanism, which is determined using the Lyapunov theory, is constructed using a sigma-modification-type updating law. The evaluation of control error in terms of the neural network learning error is performed. That is, the control error converges asymptotically to a neighborhood of zero, whose size is evaluated and depends on the approximation error of the neural network. In the design and analysis of neural network-based control systems, it is important to take into account the neural network learning error and its influence on the control error of the plant. Simulation results showing the feasibility and performance of the proposed approach are given.

A junction-tree based learning algorithm to optimize network wide traffic control: A coordinated multi-agent framework

DOE PAGES

Zhu, Feng; Aziz, H. M. Abdul; Qian, Xinwu; ...

2015-01-31

Our study develops a novel reinforcement learning algorithm for the challenging coordinated signal control problem. Traffic signals are modeled as intelligent agents interacting with the stochastic traffic environment. The model is built on the framework of coordinated reinforcement learning. The Junction Tree Algorithm (JTA) based reinforcement learning is proposed to obtain an exact inference of the best joint actions for all the coordinated intersections. Moreover, the algorithm is implemented and tested with a network containing 18 signalized intersections in VISSIM. Finally, our results show that the JTA based algorithm outperforms independent learning (Q-learning), real-time adaptive learning, and fixed timing plansmore » in terms of average delay, number of stops, and vehicular emissions at the network level.« less

An Actor-Critic based controller for glucose regulation in type 1 diabetes.

PubMed

Daskalaki, Elena; Diem, Peter; Mougiakakou, Stavroula G

2013-02-01

A novel adaptive approach for glucose control in individuals with type 1 diabetes under sensor-augmented pump therapy is proposed. The controller, is based on Actor-Critic (AC) learning and is inspired by the principles of reinforcement learning and optimal control theory. The main characteristics of the proposed controller are (i) simultaneous adjustment of both the insulin basal rate and the bolus dose, (ii) initialization based on clinical procedures, and (iii) real-time personalization. The effectiveness of the proposed algorithm in terms of glycemic control has been investigated in silico in adults, adolescents and children under open-loop and closed-loop approaches, using announced meals with uncertainties in the order of ±25% in the estimation of carbohydrates. The results show that glucose regulation is efficient in all three groups of patients, even with uncertainties in the level of carbohydrates in the meal. The percentages in the A+B zones of the Control Variability Grid Analysis (CVGA) were 100% for adults, and 93% for both adolescents and children. The AC based controller seems to be a promising approach for the automatic adjustment of insulin infusion in order to improve glycemic control. After optimization of the algorithm, the controller will be tested in a clinical trial. Copyright © 2012 Elsevier Ireland Ltd. All rights reserved.

Acetylcholine Neuromodulation in Normal and Abnormal Learning and Memory: Vigilance Control in Waking, Sleep, Autism, Amnesia and Alzheimer’s Disease

PubMed Central

Grossberg, Stephen

2017-01-01

Adaptive Resonance Theory, or ART, is a neural model that explains how normal and abnormal brains may learn to categorize and recognize objects and events in a changing world, and how these learned categories may be remembered for a long time. This article uses ART to propose and unify the explanation of diverse data about normal and abnormal modulation of learning and memory by acetylcholine (ACh). In ART, vigilance control determines whether learned categories will be general and abstract, or specific and concrete. ART models how vigilance may be regulated by ACh release in layer 5 neocortical cells by influencing after-hyperpolarization (AHP) currents. This phasic ACh release is mediated by cells in the nucleus basalis (NB) of Meynert that are activated by unexpected events. The article additionally discusses data about ACh-mediated tonic control of vigilance. ART proposes that there are often dynamic breakdowns of tonic control in mental disorders such as autism, where vigilance remains high, and medial temporal amnesia, where vigilance remains low. Tonic control also occurs during sleep-wake cycles. Properties of Up and Down states during slow wave sleep arise in ACh-modulated laminar cortical ART circuits that carry out processes in awake individuals of contrast normalization, attentional modulation, decision-making, activity-dependent habituation, and mismatch-mediated reset. These slow wave sleep circuits interact with circuits that control circadian rhythms and memory consolidation. Tonic control properties also clarify how Alzheimer’s disease symptoms follow from a massive structural degeneration that includes undermining vigilance control by ACh in cortical layers 3 and 5. Sleep disruptions before and during Alzheimer’s disease, and how they contribute to a vicious cycle of plaque formation in layers 3 and 5, are also clarified from this perspective. PMID:29163063

A Sarsa(λ)-based control model for real-time traffic light coordination.

PubMed

Zhou, Xiaoke; Zhu, Fei; Liu, Quan; Fu, Yuchen; Huang, Wei

2014-01-01

Traffic problems often occur due to the traffic demands by the outnumbered vehicles on road. Maximizing traffic flow and minimizing the average waiting time are the goals of intelligent traffic control. Each junction wants to get larger traffic flow. During the course, junctions form a policy of coordination as well as constraints for adjacent junctions to maximize their own interests. A good traffic signal timing policy is helpful to solve the problem. However, as there are so many factors that can affect the traffic control model, it is difficult to find the optimal solution. The disability of traffic light controllers to learn from past experiences caused them to be unable to adaptively fit dynamic changes of traffic flow. Considering dynamic characteristics of the actual traffic environment, reinforcement learning algorithm based traffic control approach can be applied to get optimal scheduling policy. The proposed Sarsa(λ)-based real-time traffic control optimization model can maintain the traffic signal timing policy more effectively. The Sarsa(λ)-based model gains traffic cost of the vehicle, which considers delay time, the number of waiting vehicles, and the integrated saturation from its experiences to learn and determine the optimal actions. The experiment results show an inspiring improvement in traffic control, indicating the proposed model is capable of facilitating real-time dynamic traffic control.

Biologically Inspired SNN for Robot Control.

PubMed

Nichols, Eric; McDaid, Liam J; Siddique, Nazmul

2013-02-01

This paper proposes a spiking-neural-network-based robot controller inspired by the control structures of biological systems. Information is routed through the network using facilitating dynamic synapses with short-term plasticity. Learning occurs through long-term synaptic plasticity which is implemented using the temporal difference learning rule to enable the robot to learn to associate the correct movement with the appropriate input conditions. The network self-organizes to provide memories of environments that the robot encounters. A Pioneer robot simulator with laser and sonar proximity sensors is used to verify the performance of the network with a wall-following task, and the results are presented.

Neuromorphic learning of continuous-valued mappings from noise-corrupted data

NASA Technical Reports Server (NTRS)

Troudet, T.; Merrill, W.

1991-01-01

The effect of noise on the learning performance of the backpropagation algorithm is analyzed. A selective sampling of the training set is proposed to maximize the learning of control laws by backpropagation, when the data have been corrupted by noise. The training scheme is applied to the nonlinear control of a cart-pole system in the presence of noise. The neural computation provides the neurocontroller with good noise-filtering properties. In the presence of plant noise, the neurocontroller is found to be more stable than the teacher. A novel perspective on the application of neural network technology to control engineering is presented.

Visually guided gait modifications for stepping over an obstacle: a bio-inspired approach.

PubMed

Silva, Pedro; Matos, Vitor; Santos, Cristina P

2014-02-01

There is an increasing interest in conceiving robotic systems that are able to move and act in an unstructured and not predefined environment, for which autonomy and adaptability are crucial features. In nature, animals are autonomous biological systems, which often serve as bio-inspiration models, not only for their physical and mechanical properties, but also their control structures that enable adaptability and autonomy-for which learning is (at least) partially responsible. This work proposes a system which seeks to enable a quadruped robot to online learn to detect and to avoid stumbling on an obstacle in its path. The detection relies in a forward internal model that estimates the robot's perceptive information by exploring the locomotion repetitive nature. The system adapts the locomotion in order to place the robot optimally before attempting to step over the obstacle, avoiding any stumbling. Locomotion adaptation is achieved by changing control parameters of a central pattern generator (CPG)-based locomotion controller. The mechanism learns the necessary alterations to the stride length in order to adapt the locomotion by changing the required CPG parameter. Both learning tasks occur online and together define a sensorimotor map, which enables the robot to learn to step over the obstacle in its path. Simulation results show the feasibility of the proposed approach.

Emotional Learning Based Intelligent Controllers for Rotor Flux Oriented Control of Induction Motor

NASA Astrophysics Data System (ADS)

Abdollahi, Rohollah; Farhangi, Reza; Yarahmadi, Ali

2014-08-01

This paper presents design and evaluation of a novel approach based on emotional learning to improve the speed control system of rotor flux oriented control of induction motor. The controller includes a neuro-fuzzy system with speed error and its derivative as inputs. A fuzzy critic evaluates the present situation, and provides the emotional signal (stress). The controller modifies its characteristics so that the critics stress is reduced. The comparative simulation results show that the proposed controller is more robust and hence found to be a suitable replacement of the conventional PI controller for the high performance industrial drive applications.

Online learning control using adaptive critic designs with sparse kernel machines.

PubMed

Xu, Xin; Hou, Zhongsheng; Lian, Chuanqiang; He, Haibo

2013-05-01

In the past decade, adaptive critic designs (ACDs), including heuristic dynamic programming (HDP), dual heuristic programming (DHP), and their action-dependent ones, have been widely studied to realize online learning control of dynamical systems. However, because neural networks with manually designed features are commonly used to deal with continuous state and action spaces, the generalization capability and learning efficiency of previous ACDs still need to be improved. In this paper, a novel framework of ACDs with sparse kernel machines is presented by integrating kernel methods into the critic of ACDs. To improve the generalization capability as well as the computational efficiency of kernel machines, a sparsification method based on the approximately linear dependence analysis is used. Using the sparse kernel machines, two kernel-based ACD algorithms, that is, kernel HDP (KHDP) and kernel DHP (KDHP), are proposed and their performance is analyzed both theoretically and empirically. Because of the representation learning and generalization capability of sparse kernel machines, KHDP and KDHP can obtain much better performance than previous HDP and DHP with manually designed neural networks. Simulation and experimental results of two nonlinear control problems, that is, a continuous-action inverted pendulum problem and a ball and plate control problem, demonstrate the effectiveness of the proposed kernel ACD methods.

Learning Grasp Strategies Composed of Contact Relative Motions

NASA Technical Reports Server (NTRS)

Platt, Robert, Jr.

2007-01-01

Of central importance to grasp synthesis algorithms are the assumptions made about the object to be grasped and the sensory information that is available. Many approaches avoid the issue of sensing entirely by assuming that complete information is available. In contrast, this paper proposes an approach to grasp synthesis expressed in terms of units of control that simultaneously change the contact configuration and sense information about the object and the relative manipulator-object pose. These units of control, known as contact relative motions (CRMs), allow the grasp synthesis problem to be recast as an optimal control problem where the goal is to find a strategy for executing CRMs that leads to a grasp in the shortest number of steps. An experiment is described that uses Robonaut, the NASA-JSC space humanoid, to show that CRMs are a viable means of synthesizing grasps. However, because of the limited amount of information that a single CRM can sense, the optimal control problem may be partially observable. This paper proposes expressing the problem as a k-order Markov Decision Process (MDP) and solving it using Reinforcement Learning. This approach is tested in a simulation of a two-contact manipulator that learns to grasp an object. Grasp strategies learned in simulation are tested on the physical Robonaut platform and found to lead to grasp configurations consistently.

A Novel Clustering Method Curbing the Number of States in Reinforcement Learning

NASA Astrophysics Data System (ADS)

Kotani, Naoki; Nunobiki, Masayuki; Taniguchi, Kenji

We propose an efficient state-space construction method for a reinforcement learning. Our method controls the number of categories with improving the clustering method of Fuzzy ART which is an autonomous state-space construction method. The proposed method represents weight vector as the mean value of input vectors in order to curb the number of new categories and eliminates categories whose state values are low to curb the total number of categories. As the state value is updated, the size of category becomes small to learn policy strictly. We verified the effectiveness of the proposed method with simulations of a reaching problem for a two-link robot arm. We confirmed that the number of categories was reduced and the agent achieved the complex task quickly.

Multi-Objective Reinforcement Learning-based Deep Neural Networks for Cognitive Space Communications

NASA Technical Reports Server (NTRS)

Ferreria, Paulo; Paffenroth, Randy; Wyglinski, Alexander M.; Hackett, Timothy; Bilen, Sven; Reinhart, Richard; Mortensen, Dale

2017-01-01

Future communication subsystems of space exploration missions can potentially benefit from software-defined radios (SDRs) controlled by machine learning algorithms. In this paper, we propose a novel hybrid radio resource allocation management control algorithm that integrates multi-objective reinforcement learning and deep artificial neural networks. The objective is to efficiently manage communications system resources by monitoring performance functions with common dependent variables that result in conflicting goals. The uncertainty in the performance of thousands of different possible combinations of radio parameters makes the trade-off between exploration and exploitation in reinforcement learning (RL) much more challenging for future critical space-based missions. Thus, the system should spend as little time as possible on exploring actions, and whenever it explores an action, it should perform at acceptable levels most of the time. The proposed approach enables on-line learning by interactions with the environment and restricts poor resource allocation performance through virtual environment exploration. Improvements in the multiobjective performance can be achieved via transmitter parameter adaptation on a packet-basis, with poorly predicted performance promptly resulting in rejected decisions. Simulations presented in this work considered the DVB-S2 standard adaptive transmitter parameters and additional ones expected to be present in future adaptive radio systems. Performance results are provided by analysis of the proposed hybrid algorithm when operating across a satellite communication channel from Earth to GEO orbit during clear sky conditions. The proposed approach constitutes part of the core cognitive engine proof-of-concept to be delivered to the NASA Glenn Research Center SCaN Testbed located onboard the International Space Station.

Multi-Objective Reinforcement Learning-Based Deep Neural Networks for Cognitive Space Communications

NASA Technical Reports Server (NTRS)

Ferreria, Paulo Victor R.; Paffenroth, Randy; Wyglinski, Alexander M.; Hackett, Timothy M.; Bilen, Sven G.; Reinhart, Richard C.; Mortensen, Dale J.

2017-01-01

Future communication subsystems of space exploration missions can potentially benefit from software-defined radios (SDRs) controlled by machine learning algorithms. In this paper, we propose a novel hybrid radio resource allocation management control algorithm that integrates multi-objective reinforcement learning and deep artificial neural networks. The objective is to efficiently manage communications system resources by monitoring performance functions with common dependent variables that result in conflicting goals. The uncertainty in the performance of thousands of different possible combinations of radio parameters makes the trade-off between exploration and exploitation in reinforcement learning (RL) much more challenging for future critical space-based missions. Thus, the system should spend as little time as possible on exploring actions, and whenever it explores an action, it should perform at acceptable levels most of the time. The proposed approach enables on-line learning by interactions with the environment and restricts poor resource allocation performance through virtual environment exploration. Improvements in the multiobjective performance can be achieved via transmitter parameter adaptation on a packet-basis, with poorly predicted performance promptly resulting in rejected decisions. Simulations presented in this work considered the DVB-S2 standard adaptive transmitter parameters and additional ones expected to be present in future adaptive radio systems. Performance results are provided by analysis of the proposed hybrid algorithm when operating across a satellite communication channel from Earth to GEO orbit during clear sky conditions. The proposed approach constitutes part of the core cognitive engine proof-of-concept to be delivered to the NASA Glenn Research Center SCaN Testbed located onboard the International Space Station.

Indirect adaptive fuzzy wavelet neural network with self- recurrent consequent part for AC servo system.

PubMed

Hou, Runmin; Wang, Li; Gao, Qiang; Hou, Yuanglong; Wang, Chao

2017-09-01

This paper proposes a novel indirect adaptive fuzzy wavelet neural network (IAFWNN) to control the nonlinearity, wide variations in loads, time-variation and uncertain disturbance of the ac servo system. In the proposed approach, the self-recurrent wavelet neural network (SRWNN) is employed to construct an adaptive self-recurrent consequent part for each fuzzy rule of TSK fuzzy model. For the IAFWNN controller, the online learning algorithm is based on back propagation (BP) algorithm. Moreover, an improved particle swarm optimization (IPSO) is used to adapt the learning rate. The aid of an adaptive SRWNN identifier offers the real-time gradient information to the adaptive fuzzy wavelet neural controller to overcome the impact of parameter variations, load disturbances and other uncertainties effectively, and has a good dynamic. The asymptotical stability of the system is guaranteed by using the Lyapunov method. The result of the simulation and the prototype test prove that the proposed are effective and suitable. Copyright © 2017. Published by Elsevier Ltd.

Fuzzy Counter Propagation Neural Network Control for a Class of Nonlinear Dynamical Systems

PubMed Central

Sakhre, Vandana; Jain, Sanjeev; Sapkal, Vilas S.; Agarwal, Dev P.

2015-01-01

Fuzzy Counter Propagation Neural Network (FCPN) controller design is developed, for a class of nonlinear dynamical systems. In this process, the weight connecting between the instar and outstar, that is, input-hidden and hidden-output layer, respectively, is adjusted by using Fuzzy Competitive Learning (FCL). FCL paradigm adopts the principle of learning, which is used to calculate Best Matched Node (BMN) which is proposed. This strategy offers a robust control of nonlinear dynamical systems. FCPN is compared with the existing network like Dynamic Network (DN) and Back Propagation Network (BPN) on the basis of Mean Absolute Error (MAE), Mean Square Error (MSE), Best Fit Rate (BFR), and so forth. It envisages that the proposed FCPN gives better results than DN and BPN. The effectiveness of the proposed FCPN algorithms is demonstrated through simulations of four nonlinear dynamical systems and multiple input and single output (MISO) and a single input and single output (SISO) gas furnace Box-Jenkins time series data. PMID:26366169

Fuzzy Counter Propagation Neural Network Control for a Class of Nonlinear Dynamical Systems.

PubMed

Sakhre, Vandana; Jain, Sanjeev; Sapkal, Vilas S; Agarwal, Dev P

2015-01-01

Fuzzy Counter Propagation Neural Network (FCPN) controller design is developed, for a class of nonlinear dynamical systems. In this process, the weight connecting between the instar and outstar, that is, input-hidden and hidden-output layer, respectively, is adjusted by using Fuzzy Competitive Learning (FCL). FCL paradigm adopts the principle of learning, which is used to calculate Best Matched Node (BMN) which is proposed. This strategy offers a robust control of nonlinear dynamical systems. FCPN is compared with the existing network like Dynamic Network (DN) and Back Propagation Network (BPN) on the basis of Mean Absolute Error (MAE), Mean Square Error (MSE), Best Fit Rate (BFR), and so forth. It envisages that the proposed FCPN gives better results than DN and BPN. The effectiveness of the proposed FCPN algorithms is demonstrated through simulations of four nonlinear dynamical systems and multiple input and single output (MISO) and a single input and single output (SISO) gas furnace Box-Jenkins time series data.

Brain computer interface learning for systems based on electrocorticography and intracortical microelectrode arrays.

PubMed

Hiremath, Shivayogi V; Chen, Weidong; Wang, Wei; Foldes, Stephen; Yang, Ying; Tyler-Kabara, Elizabeth C; Collinger, Jennifer L; Boninger, Michael L

2015-01-01

A brain-computer interface (BCI) system transforms neural activity into control signals for external devices in real time. A BCI user needs to learn to generate specific cortical activity patterns to control external devices effectively. We call this process BCI learning, and it often requires significant effort and time. Therefore, it is important to study this process and develop novel and efficient approaches to accelerate BCI learning. This article reviews major approaches that have been used for BCI learning, including computer-assisted learning, co-adaptive learning, operant conditioning, and sensory feedback. We focus on BCIs based on electrocorticography and intracortical microelectrode arrays for restoring motor function. This article also explores the possibility of brain modulation techniques in promoting BCI learning, such as electrical cortical stimulation, transcranial magnetic stimulation, and optogenetics. Furthermore, as proposed by recent BCI studies, we suggest that BCI learning is in many ways analogous to motor and cognitive skill learning, and therefore skill learning should be a useful metaphor to model BCI learning.

Machine Learning Methods for Production Cases Analysis

NASA Astrophysics Data System (ADS)

Mokrova, Nataliya V.; Mokrov, Alexander M.; Safonova, Alexandra V.; Vishnyakov, Igor V.

2018-03-01

Approach to analysis of events occurring during the production process were proposed. Described machine learning system is able to solve classification tasks related to production control and hazard identification at an early stage. Descriptors of the internal production network data were used for training and testing of applied models. k-Nearest Neighbors and Random forest methods were used to illustrate and analyze proposed solution. The quality of the developed classifiers was estimated using standard statistical metrics, such as precision, recall and accuracy.

Robust leader-follower formation tracking control of multiple underactuated surface vessels

NASA Astrophysics Data System (ADS)

Peng, Zhou-hua; Wang, Dan; Lan, Wei-yao; Sun, Gang

2012-09-01

This paper is concerned with the formation control problem of multiple underactuated surface vessels moving in a leader-follower formation. The formation is achieved by the follower to track a virtual target defined relative to the leader. A robust adaptive target tracking law is proposed by using neural network and backstepping techniques. The advantage of the proposed control scheme is that the uncertain nonlinear dynamics caused by Coriolis/centripetal forces, nonlinear damping, unmodeled hydrodynamics and disturbances from the environment can be compensated by on line learning. Based on Lyapunov analysis, the proposed controller guarantees the tracking errors converge to a small neighborhood of the origin. Simulation results demonstrate the effectiveness of the control strategy.

Realization of Comfortable Massage by Using Iterative Learning Control Based on EEG

NASA Astrophysics Data System (ADS)

Teramae, Tatsuya; Kushida, Daisuke; Takemori, Fumiaki; Kitamura, Akira

Recently the massage chair is used by a lot of people because they are able to use it easily at home. However a present massage chair only realizes the massage motion. Moreover the massage chair can not consider the user’s condition and massage force. On the other hand, the professional masseur is according to presume the mental condition by patient’s reaction. Then this paper proposes the method of applying masseur’s procedure for the massage chair using iterative learning control based on EEG. And massage force is estimated by acceleration sensor. The realizability of the proposed method is verified by the experimental works using the massage chair.

Building Environment Analysis based on Temperature and Humidity for Smart Energy Systems

PubMed Central

Yun, Jaeseok; Won, Kwang-Ho

2012-01-01

In this paper, we propose a new HVAC (heating, ventilation, and air conditioning) control strategy as part of the smart energy system that can balance occupant comfort against building energy consumption using ubiquitous sensing and machine learning technology. We have developed ZigBee-based wireless sensor nodes and collected realistic temperature and humidity data during one month from a laboratory environment. With the collected data, we have established a building environment model using machine learning algorithms, which can be used to assess occupant comfort level. We expect the proposed HVAC control strategy will be able to provide occupants with a consistently comfortable working or home environment. PMID:23202004

A composite self tuning strategy for fuzzy control of dynamic systems

NASA Technical Reports Server (NTRS)

Shieh, C.-Y.; Nair, Satish S.

1992-01-01

The feature of self learning makes fuzzy logic controllers attractive in control applications. This paper proposes a strategy to tune the fuzzy logic controller on-line by tuning the data base as well as the rule base. The structure of the controller is outlined and preliminary results are presented using simulation studies.

Quantum Ensemble Classification: A Sampling-Based Learning Control Approach.

PubMed

Chen, Chunlin; Dong, Daoyi; Qi, Bo; Petersen, Ian R; Rabitz, Herschel

2017-06-01

Quantum ensemble classification (QEC) has significant applications in discrimination of atoms (or molecules), separation of isotopes, and quantum information extraction. However, quantum mechanics forbids deterministic discrimination among nonorthogonal states. The classification of inhomogeneous quantum ensembles is very challenging, since there exist variations in the parameters characterizing the members within different classes. In this paper, we recast QEC as a supervised quantum learning problem. A systematic classification methodology is presented by using a sampling-based learning control (SLC) approach for quantum discrimination. The classification task is accomplished via simultaneously steering members belonging to different classes to their corresponding target states (e.g., mutually orthogonal states). First, a new discrimination method is proposed for two similar quantum systems. Then, an SLC method is presented for QEC. Numerical results demonstrate the effectiveness of the proposed approach for the binary classification of two-level quantum ensembles and the multiclass classification of multilevel quantum ensembles.

Real time eye tracking using Kalman extended spatio-temporal context learning

NASA Astrophysics Data System (ADS)

Munir, Farzeen; Minhas, Fayyaz ul Amir Asfar; Jalil, Abdul; Jeon, Moongu

2017-06-01

Real time eye tracking has numerous applications in human computer interaction such as a mouse cursor control in a computer system. It is useful for persons with muscular or motion impairments. However, tracking the movement of the eye is complicated by occlusion due to blinking, head movement, screen glare, rapid eye movements, etc. In this work, we present the algorithmic and construction details of a real time eye tracking system. Our proposed system is an extension of Spatio-Temporal context learning through Kalman Filtering. Spatio-Temporal Context Learning offers state of the art accuracy in general object tracking but its performance suffers due to object occlusion. Addition of the Kalman filter allows the proposed method to model the dynamics of the motion of the eye and provide robust eye tracking in cases of occlusion. We demonstrate the effectiveness of this tracking technique by controlling the computer cursor in real time by eye movements.

Development of Advanced Verification and Validation Procedures and Tools for the Certification of Learning Systems in Aerospace Applications

NASA Technical Reports Server (NTRS)

Jacklin, Stephen; Schumann, Johann; Gupta, Pramod; Richard, Michael; Guenther, Kurt; Soares, Fola

2005-01-01

Adaptive control technologies that incorporate learning algorithms have been proposed to enable automatic flight control and vehicle recovery, autonomous flight, and to maintain vehicle performance in the face of unknown, changing, or poorly defined operating environments. In order for adaptive control systems to be used in safety-critical aerospace applications, they must be proven to be highly safe and reliable. Rigorous methods for adaptive software verification and validation must be developed to ensure that control system software failures will not occur. Of central importance in this regard is the need to establish reliable methods that guarantee convergent learning, rapid convergence (learning) rate, and algorithm stability. This paper presents the major problems of adaptive control systems that use learning to improve performance. The paper then presents the major procedures and tools presently developed or currently being developed to enable the verification, validation, and ultimate certification of these adaptive control systems. These technologies include the application of automated program analysis methods, techniques to improve the learning process, analytical methods to verify stability, methods to automatically synthesize code, simulation and test methods, and tools to provide on-line software assurance.

Model-Free Optimal Tracking Control via Critic-Only Q-Learning.

PubMed

Luo, Biao; Liu, Derong; Huang, Tingwen; Wang, Ding

2016-10-01

Model-free control is an important and promising topic in control fields, which has attracted extensive attention in the past few years. In this paper, we aim to solve the model-free optimal tracking control problem of nonaffine nonlinear discrete-time systems. A critic-only Q-learning (CoQL) method is developed, which learns the optimal tracking control from real system data, and thus avoids solving the tracking Hamilton-Jacobi-Bellman equation. First, the Q-learning algorithm is proposed based on the augmented system, and its convergence is established. Using only one neural network for approximating the Q-function, the CoQL method is developed to implement the Q-learning algorithm. Furthermore, the convergence of the CoQL method is proved with the consideration of neural network approximation error. With the convergent Q-function obtained from the CoQL method, the adaptive optimal tracking control is designed based on the gradient descent scheme. Finally, the effectiveness of the developed CoQL method is demonstrated through simulation studies. The developed CoQL method learns with off-policy data and implements with a critic-only structure, thus it is easy to realize and overcome the inadequate exploration problem.

Cost-Benefit Arbitration Between Multiple Reinforcement-Learning Systems.

PubMed

Kool, Wouter; Gershman, Samuel J; Cushman, Fiery A

2017-09-01

Human behavior is sometimes determined by habit and other times by goal-directed planning. Modern reinforcement-learning theories formalize this distinction as a competition between a computationally cheap but inaccurate model-free system that gives rise to habits and a computationally expensive but accurate model-based system that implements planning. It is unclear, however, how people choose to allocate control between these systems. Here, we propose that arbitration occurs by comparing each system's task-specific costs and benefits. To investigate this proposal, we conducted two experiments showing that people increase model-based control when it achieves greater accuracy than model-free control, and especially when the rewards of accurate performance are amplified. In contrast, they are insensitive to reward amplification when model-based and model-free control yield equivalent accuracy. This suggests that humans adaptively balance habitual and planned action through on-line cost-benefit analysis.

Redundancy Matters: Flexible Learning of Multiple Contingencies in Infants

ERIC Educational Resources Information Center

Sloutsky, Vladimir M.; Robinson, Christopher W.

2013-01-01

Many objects and events can be categorized in different ways, and learning multiple categories in parallel often requires flexibly attending to different stimulus dimensions in different contexts. Although infants and young children often exhibit poor attentional control, several theoretical proposals argue that such flexibility can be achieved…

Evidencing Learning Outcomes: A Multi-Level, Multi-Dimensional Course Alignment Model

ERIC Educational Resources Information Center

Sridharan, Bhavani; Leitch, Shona; Watty, Kim

2015-01-01

This conceptual framework proposes a multi-level, multi-dimensional course alignment model to implement a contextualised constructive alignment of rubric design that authentically evidences and assesses learning outcomes. By embedding quality control mechanisms at each level for each dimension, this model facilitates the development of an aligned…

Lifelong Learning for the 21st Century.

ERIC Educational Resources Information Center

Goodnight, Ron

The Lifelong Learning Center for the 21st Century was proposed to provide personal renewal and technical training for employees at a major United States automotive manufacturing company when it implemented a new, computer-based Computer Numerical Controlled (CNC) machining, robotics, and high technology facility. The employees needed training for…

76 FR 27325 - Proposed Data Collections Submitted for Public Comment and Recommendations

Federal Register 2010, 2011, 2012, 2013, 2014

2011-05-11

... Enhanced Implementation of the ``Learn the Signs. Act Early.'' Campaign in 4 Target Sites,--New--National Center on Birth Defects and Developmental Disabilities (NCBDDD), Centers for Disease Control and Prevention (CDC). Background and Brief Description CDC's ``Learn the Signs Act Early'' campaign is a health...

Fault-tolerant nonlinear adaptive flight control using sliding mode online learning.

PubMed

Krüger, Thomas; Schnetter, Philipp; Placzek, Robin; Vörsmann, Peter

2012-08-01

An expanded nonlinear model inversion flight control strategy using sliding mode online learning for neural networks is presented. The proposed control strategy is implemented for a small unmanned aircraft system (UAS). This class of aircraft is very susceptible towards nonlinearities like atmospheric turbulence, model uncertainties and of course system failures. Therefore, these systems mark a sensible testbed to evaluate fault-tolerant, adaptive flight control strategies. Within this work the concept of feedback linearization is combined with feed forward neural networks to compensate for inversion errors and other nonlinear effects. Backpropagation-based adaption laws of the network weights are used for online training. Within these adaption laws the standard gradient descent backpropagation algorithm is augmented with the concept of sliding mode control (SMC). Implemented as a learning algorithm, this nonlinear control strategy treats the neural network as a controlled system and allows a stable, dynamic calculation of the learning rates. While considering the system's stability, this robust online learning method therefore offers a higher speed of convergence, especially in the presence of external disturbances. The SMC-based flight controller is tested and compared with the standard gradient descent backpropagation algorithm in the presence of system failures. Copyright © 2012 Elsevier Ltd. All rights reserved.

Scheduled power tracking control of the wind-storage hybrid system based on the reinforcement learning theory

NASA Astrophysics Data System (ADS)

Li, Ze

2017-09-01

In allusion to the intermittency and uncertainty of the wind electricity, energy storage and wind generator are combined into a hybrid system to improve the controllability of the output power. A scheduled power tracking control method is proposed based on the reinforcement learning theory and Q-learning algorithm. In this method, the state space of the environment is formed with two key factors, i.e. the state of charge of the energy storage and the difference value between the actual wind power and scheduled power, the feasible action is the output power of the energy storage, and the corresponding immediate rewarding function is designed to reflect the rationality of the control action. By interacting with the environment and learning from the immediate reward, the optimal control strategy is gradually formed. After that, it could be applied to the scheduled power tracking control of the hybrid system. Finally, the rationality and validity of the method are verified through simulation examples.

Impact of Behavioral Control on the Processing of Nociceptive Stimulation

PubMed Central

Grau, James W.; Huie, J. Russell; Garraway, Sandra M.; Hook, Michelle A.; Crown, Eric D.; Baumbauer, Kyle M.; Lee, Kuan H.; Hoy, Kevin C.; Ferguson, Adam R.

2012-01-01

How nociceptive signals are processed within the spinal cord, and whether these signals lead to behavioral signs of neuropathic pain, depends upon their relation to other events and behavior. Our work shows that these relations can have a lasting effect on spinal plasticity, inducing a form of learning that alters the effect of subsequent nociceptive stimuli. The capacity of lower spinal systems to adapt, in the absence of brain input, is examined in spinally transected rats that receive a nociceptive shock to the tibialis anterior muscle of one hind leg. If shock is delivered whenever the leg is extended (controllable stimulation), it induces an increase in flexion duration that minimizes net shock exposure. This learning is not observed in subjects that receive the same amount of shock independent of leg position (uncontrollable stimulation). These two forms of stimulation have a lasting, and divergent, effect on subsequent learning: controllable stimulation enables learning whereas uncontrollable stimulation disables it (learning deficit). Uncontrollable stimulation also enhances mechanical reactivity. We review evidence that training with controllable stimulation engages a brain-derived neurotrophic factor (BDNF)-dependent process that can both prevent and reverse the consequences of uncontrollable shock. We relate these effects to changes in BDNF protein and TrkB signaling. Controllable stimulation is also shown to counter the effects of peripheral inflammation (from intradermal capsaicin). A model is proposed that assumes nociceptive input is gated at an early sensory stage. This gate is sensitive to current environmental relations (between proprioceptive and nociceptive input), allowing stimulation to be classified as controllable or uncontrollable. We further propose that the status of this gate is affected by past experience and that a history of uncontrollable stimulation will promote the development of neuropathic pain. PMID:22934018

A Sarsa(λ)-Based Control Model for Real-Time Traffic Light Coordination

PubMed Central

Zhu, Fei; Liu, Quan; Fu, Yuchen; Huang, Wei

2014-01-01

Traffic problems often occur due to the traffic demands by the outnumbered vehicles on road. Maximizing traffic flow and minimizing the average waiting time are the goals of intelligent traffic control. Each junction wants to get larger traffic flow. During the course, junctions form a policy of coordination as well as constraints for adjacent junctions to maximize their own interests. A good traffic signal timing policy is helpful to solve the problem. However, as there are so many factors that can affect the traffic control model, it is difficult to find the optimal solution. The disability of traffic light controllers to learn from past experiences caused them to be unable to adaptively fit dynamic changes of traffic flow. Considering dynamic characteristics of the actual traffic environment, reinforcement learning algorithm based traffic control approach can be applied to get optimal scheduling policy. The proposed Sarsa(λ)-based real-time traffic control optimization model can maintain the traffic signal timing policy more effectively. The Sarsa(λ)-based model gains traffic cost of the vehicle, which considers delay time, the number of waiting vehicles, and the integrated saturation from its experiences to learn and determine the optimal actions. The experiment results show an inspiring improvement in traffic control, indicating the proposed model is capable of facilitating real-time dynamic traffic control. PMID:24592183

Adolescent Learning in the Zoo: Embedding a Non-Formal Learning Environment to Teach Formal Aspects of Vertebrate Biology

NASA Astrophysics Data System (ADS)

Randler, Christoph; Kummer, Barbara; Wilhelm, Christian

2012-06-01

The aim of this study was to assess the outcome of a zoo visit in terms of learning and retention of knowledge concerning the adaptations and behavior of vertebrate species. Basis of the work was the concept of implementing zoo visits as an out-of-school setting for formal, curriculum based learning. Our theoretical framework centers on the self-determination theory, therefore, we used a group-based, hands-on learning environment. To address this questions, we used a treatment—control design (BACI) with different treatments and a control group. Pre-, post- and retention tests were applied. All treatments led to a substantial increase of learning and retention knowledge compared to the control group. Immediately after the zoo visit, the zoo-guide tour provided the highest scores, while after a delay of 6 weeks, the learner-centered environment combined with a teacher-guided summarizing scored best. We suggest incorporating the zoo as an out-of-school environment into formal school learning, and we propose different methods to improve learning in zoo settings.

Enhanced Data Representation by Kernel Metric Learning for Dementia Diagnosis

PubMed Central

Cárdenas-Peña, David; Collazos-Huertas, Diego; Castellanos-Dominguez, German

2017-01-01

Alzheimer's disease (AD) is the kind of dementia that affects the most people around the world. Therefore, an early identification supporting effective treatments is required to increase the life quality of a wide number of patients. Recently, computer-aided diagnosis tools for dementia using Magnetic Resonance Imaging scans have been successfully proposed to discriminate between patients with AD, mild cognitive impairment, and healthy controls. Most of the attention has been given to the clinical data, provided by initiatives as the ADNI, supporting reliable researches on intervention, prevention, and treatments of AD. Therefore, there is a need for improving the performance of classification machines. In this paper, we propose a kernel framework for learning metrics that enhances conventional machines and supports the diagnosis of dementia. Our framework aims at building discriminative spaces through the maximization of center kernel alignment function, aiming at improving the discrimination of the three considered neurological classes. The proposed metric learning performance is evaluated on the widely-known ADNI database using three supervised classification machines (k-nn, SVM and NNs) for multi-class and bi-class scenarios from structural MRIs. Specifically, from ADNI collection 286 AD patients, 379 MCI patients and 231 healthy controls are used for development and validation of our proposed metric learning framework. For the experimental validation, we split the data into two subsets: 30% of subjects used like a blindfolded assessment and 70% employed for parameter tuning. Then, in the preprocessing stage, each structural MRI scan a total of 310 morphological measurements are automatically extracted from by FreeSurfer software package and concatenated to build an input feature matrix. Obtained test performance results, show that including a supervised metric learning improves the compared baseline classifiers in both scenarios. In the multi-class scenario, we achieve the best performance (accuracy 60.1%) for pretrained 1-layered NN, and we obtain measures over 90% in the average for HC vs. AD task. From the machine learning point of view, our proposal enhances the classifier performance by building spaces with a better class separability. From the clinical application, our enhancement results in a more balanced performance in each class than the compared approaches from the CADDementia challenge by increasing the sensitivity of pathological groups and the specificity of healthy controls. PMID:28798659

Adaptive Control of Exoskeleton Robots for Periodic Assistive Behaviours Based on EMG Feedback Minimisation.

PubMed

Peternel, Luka; Noda, Tomoyuki; Petrič, Tadej; Ude, Aleš; Morimoto, Jun; Babič, Jan

2016-01-01

In this paper we propose an exoskeleton control method for adaptive learning of assistive joint torque profiles in periodic tasks. We use human muscle activity as feedback to adapt the assistive joint torque behaviour in a way that the muscle activity is minimised. The user can then relax while the exoskeleton takes over the task execution. If the task is altered and the existing assistive behaviour becomes inadequate, the exoskeleton gradually adapts to the new task execution so that the increased muscle activity caused by the new desired task can be reduced. The advantage of the proposed method is that it does not require biomechanical or dynamical models. Our proposed learning system uses Dynamical Movement Primitives (DMPs) as a trajectory generator and parameters of DMPs are modulated using Locally Weighted Regression. Then, the learning system is combined with adaptive oscillators that determine the phase and frequency of motion according to measured Electromyography (EMG) signals. We tested the method with real robot experiments where subjects wearing an elbow exoskeleton had to move an object of an unknown mass according to a predefined reference motion. We further evaluated the proposed approach on a whole-arm exoskeleton to show that it is able to adaptively derive assistive torques even for multiple-joint motion.

Adaptive Control of Exoskeleton Robots for Periodic Assistive Behaviours Based on EMG Feedback Minimisation

PubMed Central

Peternel, Luka; Noda, Tomoyuki; Petrič, Tadej; Ude, Aleš; Morimoto, Jun; Babič, Jan

2016-01-01

In this paper we propose an exoskeleton control method for adaptive learning of assistive joint torque profiles in periodic tasks. We use human muscle activity as feedback to adapt the assistive joint torque behaviour in a way that the muscle activity is minimised. The user can then relax while the exoskeleton takes over the task execution. If the task is altered and the existing assistive behaviour becomes inadequate, the exoskeleton gradually adapts to the new task execution so that the increased muscle activity caused by the new desired task can be reduced. The advantage of the proposed method is that it does not require biomechanical or dynamical models. Our proposed learning system uses Dynamical Movement Primitives (DMPs) as a trajectory generator and parameters of DMPs are modulated using Locally Weighted Regression. Then, the learning system is combined with adaptive oscillators that determine the phase and frequency of motion according to measured Electromyography (EMG) signals. We tested the method with real robot experiments where subjects wearing an elbow exoskeleton had to move an object of an unknown mass according to a predefined reference motion. We further evaluated the proposed approach on a whole-arm exoskeleton to show that it is able to adaptively derive assistive torques even for multiple-joint motion. PMID:26881743

Kernel-based least squares policy iteration for reinforcement learning.

PubMed

Xu, Xin; Hu, Dewen; Lu, Xicheng

2007-07-01

In this paper, we present a kernel-based least squares policy iteration (KLSPI) algorithm for reinforcement learning (RL) in large or continuous state spaces, which can be used to realize adaptive feedback control of uncertain dynamic systems. By using KLSPI, near-optimal control policies can be obtained without much a priori knowledge on dynamic models of control plants. In KLSPI, Mercer kernels are used in the policy evaluation of a policy iteration process, where a new kernel-based least squares temporal-difference algorithm called KLSTD-Q is proposed for efficient policy evaluation. To keep the sparsity and improve the generalization ability of KLSTD-Q solutions, a kernel sparsification procedure based on approximate linear dependency (ALD) is performed. Compared to the previous works on approximate RL methods, KLSPI makes two progresses to eliminate the main difficulties of existing results. One is the better convergence and (near) optimality guarantee by using the KLSTD-Q algorithm for policy evaluation with high precision. The other is the automatic feature selection using the ALD-based kernel sparsification. Therefore, the KLSPI algorithm provides a general RL method with generalization performance and convergence guarantee for large-scale Markov decision problems (MDPs). Experimental results on a typical RL task for a stochastic chain problem demonstrate that KLSPI can consistently achieve better learning efficiency and policy quality than the previous least squares policy iteration (LSPI) algorithm. Furthermore, the KLSPI method was also evaluated on two nonlinear feedback control problems, including a ship heading control problem and the swing up control of a double-link underactuated pendulum called acrobot. Simulation results illustrate that the proposed method can optimize controller performance using little a priori information of uncertain dynamic systems. It is also demonstrated that KLSPI can be applied to online learning control by incorporating an initial controller to ensure online performance.

Attention control learning in the decision space using state estimation

NASA Astrophysics Data System (ADS)

Gharaee, Zahra; Fatehi, Alireza; Mirian, Maryam S.; Nili Ahmadabadi, Majid

2016-05-01

The main goal of this paper is modelling attention while using it in efficient path planning of mobile robots. The key challenge in concurrently aiming these two goals is how to make an optimal, or near-optimal, decision in spite of time and processing power limitations, which inherently exist in a typical multi-sensor real-world robotic application. To efficiently recognise the environment under these two limitations, attention of an intelligent agent is controlled by employing the reinforcement learning framework. We propose an estimation method using estimated mixture-of-experts task and attention learning in perceptual space. An agent learns how to employ its sensory resources, and when to stop observing, by estimating its perceptual space. In this paper, static estimation of the state space in a learning task problem, which is examined in the WebotsTM simulator, is performed. Simulation results show that a robot learns how to achieve an optimal policy with a controlled cost by estimating the state space instead of continually updating sensory information.

Machine learning enhanced optical distance sensor

NASA Astrophysics Data System (ADS)

Amin, M. Junaid; Riza, N. A.

2018-01-01

Presented for the first time is a machine learning enhanced optical distance sensor. The distance sensor is based on our previously demonstrated distance measurement technique that uses an Electronically Controlled Variable Focus Lens (ECVFL) with a laser source to illuminate a target plane with a controlled optical beam spot. This spot with varying spot sizes is viewed by an off-axis camera and the spot size data is processed to compute the distance. In particular, proposed and demonstrated in this paper is the use of a regularized polynomial regression based supervised machine learning algorithm to enhance the accuracy of the operational sensor. The algorithm uses the acquired features and corresponding labels that are the actual target distance values to train a machine learning model. The optimized training model is trained over a 1000 mm (or 1 m) experimental target distance range. Using the machine learning algorithm produces a training set and testing set distance measurement errors of <0.8 mm and <2.2 mm, respectively. The test measurement error is at least a factor of 4 improvement over our prior sensor demonstration without the use of machine learning. Applications for the proposed sensor include industrial scenario distance sensing where target material specific training models can be generated to realize low <1% measurement error distance measurements.

Semi-supervised manifold learning with affinity regularization for Alzheimer's disease identification using positron emission tomography imaging.

PubMed

Lu, Shen; Xia, Yong; Cai, Tom Weidong; Feng, David Dagan

2015-01-01

Dementia, Alzheimer's disease (AD) in particular is a global problem and big threat to the aging population. An image based computer-aided dementia diagnosis method is needed to providing doctors help during medical image examination. Many machine learning based dementia classification methods using medical imaging have been proposed and most of them achieve accurate results. However, most of these methods make use of supervised learning requiring fully labeled image dataset, which usually is not practical in real clinical environment. Using large amount of unlabeled images can improve the dementia classification performance. In this study we propose a new semi-supervised dementia classification method based on random manifold learning with affinity regularization. Three groups of spatial features are extracted from positron emission tomography (PET) images to construct an unsupervised random forest which is then used to regularize the manifold learning objective function. The proposed method, stat-of-the-art Laplacian support vector machine (LapSVM) and supervised SVM are applied to classify AD and normal controls (NC). The experiment results show that learning with unlabeled images indeed improves the classification performance. And our method outperforms LapSVM on the same dataset.

An open-closed-loop iterative learning control approach for nonlinear switched systems with application to freeway traffic control

NASA Astrophysics Data System (ADS)

Sun, Shu-Ting; Li, Xiao-Dong; Zhong, Ren-Xin

2017-10-01

For nonlinear switched discrete-time systems with input constraints, this paper presents an open-closed-loop iterative learning control (ILC) approach, which includes a feedforward ILC part and a feedback control part. Under a given switching rule, the mathematical induction is used to prove the convergence of ILC tracking error in each subsystem. It is demonstrated that the convergence of ILC tracking error is dependent on the feedforward control gain, but the feedback control can speed up the convergence process of ILC by a suitable selection of feedback control gain. A switched freeway traffic system is used to illustrate the effectiveness of the proposed ILC law.

Robust representation and recognition of facial emotions using extreme sparse learning.

PubMed

Shojaeilangari, Seyedehsamaneh; Yau, Wei-Yun; Nandakumar, Karthik; Li, Jun; Teoh, Eam Khwang

2015-07-01

Recognition of natural emotions from human faces is an interesting topic with a wide range of potential applications, such as human-computer interaction, automated tutoring systems, image and video retrieval, smart environments, and driver warning systems. Traditionally, facial emotion recognition systems have been evaluated on laboratory controlled data, which is not representative of the environment faced in real-world applications. To robustly recognize the facial emotions in real-world natural situations, this paper proposes an approach called extreme sparse learning, which has the ability to jointly learn a dictionary (set of basis) and a nonlinear classification model. The proposed approach combines the discriminative power of extreme learning machine with the reconstruction property of sparse representation to enable accurate classification when presented with noisy signals and imperfect data recorded in natural settings. In addition, this paper presents a new local spatio-temporal descriptor that is distinctive and pose-invariant. The proposed framework is able to achieve the state-of-the-art recognition accuracy on both acted and spontaneous facial emotion databases.

Distributed Economic Dispatch in Microgrids Based on Cooperative Reinforcement Learning.

PubMed

Liu, Weirong; Zhuang, Peng; Liang, Hao; Peng, Jun; Huang, Zhiwu; Weirong Liu; Peng Zhuang; Hao Liang; Jun Peng; Zhiwu Huang; Liu, Weirong; Liang, Hao; Peng, Jun; Zhuang, Peng; Huang, Zhiwu

2018-06-01

Microgrids incorporated with distributed generation (DG) units and energy storage (ES) devices are expected to play more and more important roles in the future power systems. Yet, achieving efficient distributed economic dispatch in microgrids is a challenging issue due to the randomness and nonlinear characteristics of DG units and loads. This paper proposes a cooperative reinforcement learning algorithm for distributed economic dispatch in microgrids. Utilizing the learning algorithm can avoid the difficulty of stochastic modeling and high computational complexity. In the cooperative reinforcement learning algorithm, the function approximation is leveraged to deal with the large and continuous state spaces. And a diffusion strategy is incorporated to coordinate the actions of DG units and ES devices. Based on the proposed algorithm, each node in microgrids only needs to communicate with its local neighbors, without relying on any centralized controllers. Algorithm convergence is analyzed, and simulations based on real-world meteorological and load data are conducted to validate the performance of the proposed algorithm.

Molecular substrates of action control in cortico-striatal circuits.

PubMed

Shiflett, Michael W; Balleine, Bernard W

2011-09-15

The purpose of this review is to describe the molecular mechanisms in the striatum that mediate reward-based learning and action control during instrumental conditioning. Experiments assessing the neural bases of instrumental conditioning have uncovered functional circuits in the striatum, including dorsal and ventral striatal sub-regions, involved in action-outcome learning, stimulus-response learning, and the motivational control of action by reward-associated cues. Integration of dopamine (DA) and glutamate neurotransmission within these striatal sub-regions is hypothesized to enable learning and action control through its role in shaping synaptic plasticity and cellular excitability. The extracellular signal regulated kinase (ERK) appears to be particularly important for reward-based learning and action control due to its sensitivity to combined DA and glutamate receptor activation and its involvement in a range of cellular functions. ERK activation in striatal neurons is proposed to have a dual role in both the learning and performance factors that contribute to instrumental conditioning through its regulation of plasticity-related transcription factors and its modulation of intrinsic cellular excitability. Furthermore, perturbation of ERK activation by drugs of abuse may give rise to behavioral disorders such as addiction. Copyright © 2011 Elsevier Ltd. All rights reserved.

Integration Method of Emphatic Motions and Adverbial Expressions with Scalar Parameters for Robotic Motion Coaching System

NASA Astrophysics Data System (ADS)

Okuno, Keisuke; Inamura, Tetsunari

A robotic coaching system can improve humans' learning performance of motions by intelligent usage of emphatic motions and adverbial expressions according to user reactions. In robotics, however, method to control both the motions and the expressions and how to bind them had not been adequately discussed from an engineering point of view. In this paper, we propose a method for controlling and binding emphatic motions and adverbial expressions by using two scalar parameters in a phase space. In the phase space, variety of motion patterns and verbal expressions are connected and can be expressed as static points. We show the feasibility of the proposing method through experiments of actual sport coaching tasks for beginners. From the results of participants' improvements in motion learning, we confirmed the feasibility of the methods to control and bind emphatic motions and adverbial expressions, as well as confirmed contribution of the emphatic motions and positive correlation of adverbial expressions for participants' improvements in motion learning. Based on the results, we introduce a hypothesis that individually optimized method for binding adverbial expression is required.

Intelligent tuning method of PID parameters based on iterative learning control for atomic force microscopy.

PubMed

Liu, Hui; Li, Yingzi; Zhang, Yingxu; Chen, Yifu; Song, Zihang; Wang, Zhenyu; Zhang, Suoxin; Qian, Jianqiang

2018-01-01

Proportional-integral-derivative (PID) parameters play a vital role in the imaging process of an atomic force microscope (AFM). Traditional parameter tuning methods require a lot of manpower and it is difficult to set PID parameters in unattended working environments. In this manuscript, an intelligent tuning method of PID parameters based on iterative learning control is proposed to self-adjust PID parameters of the AFM according to the sample topography. This method gets enough information about the output signals of PID controller and tracking error, which will be used to calculate the proper PID parameters, by repeated line scanning until convergence before normal scanning to learn the topography. Subsequently, the appropriate PID parameters are obtained by fitting method and then applied to the normal scanning process. The feasibility of the method is demonstrated by the convergence analysis. Simulations and experimental results indicate that the proposed method can intelligently tune PID parameters of the AFM for imaging different topographies and thus achieve good tracking performance. Copyright © 2017 Elsevier Ltd. All rights reserved.

Power in the Classroom VI: Verbal Control Strategies, Nonverbal Immediacy and Affective Learning.

ERIC Educational Resources Information Center

Plax, Timothy G.; And Others

Recognizing that nonverbal behaviors typically provide the framework for interpreting verbal messages, this project (the sixth in a series of projects designed to examine teacher power in the classroom) proposed and sequentially tested a heuristic model of student affective learning as a function of behavior alteration techniques and teacher…

Adaptive critic autopilot design of bank-to-turn missiles using fuzzy basis function networks.

PubMed

Lin, Chuan-Kai

2005-04-01

A new adaptive critic autopilot design for bank-to-turn missiles is presented. In this paper, the architecture of adaptive critic learning scheme contains a fuzzy-basis-function-network based associative search element (ASE), which is employed to approximate nonlinear and complex functions of bank-to-turn missiles, and an adaptive critic element (ACE) generating the reinforcement signal to tune the associative search element. In the design of the adaptive critic autopilot, the control law receives signals from a fixed gain controller, an ASE and an adaptive robust element, which can eliminate approximation errors and disturbances. Traditional adaptive critic reinforcement learning is the problem faced by an agent that must learn behavior through trial-and-error interactions with a dynamic environment, however, the proposed tuning algorithm can significantly shorten the learning time by online tuning all parameters of fuzzy basis functions and weights of ASE and ACE. Moreover, the weight updating law derived from the Lyapunov stability theory is capable of guaranteeing both tracking performance and stability. Computer simulation results confirm the effectiveness of the proposed adaptive critic autopilot.

Vibration control of building structures using self-organizing and self-learning neural networks

NASA Astrophysics Data System (ADS)

Madan, Alok

2005-11-01

Past research in artificial intelligence establishes that artificial neural networks (ANN) are effective and efficient computational processors for performing a variety of tasks including pattern recognition, classification, associative recall, combinatorial problem solving, adaptive control, multi-sensor data fusion, noise filtering and data compression, modelling and forecasting. The paper presents a potentially feasible approach for training ANN in active control of earthquake-induced vibrations in building structures without the aid of teacher signals (i.e. target control forces). A counter-propagation neural network is trained to output the control forces that are required to reduce the structural vibrations in the absence of any feedback on the correctness of the output control forces (i.e. without any information on the errors in output activations of the network). The present study shows that, in principle, the counter-propagation network (CPN) can learn from the control environment to compute the required control forces without the supervision of a teacher (unsupervised learning). Simulated case studies are presented to demonstrate the feasibility of implementing the unsupervised learning approach in ANN for effective vibration control of structures under the influence of earthquake ground motions. The proposed learning methodology obviates the need for developing a mathematical model of structural dynamics or training a separate neural network to emulate the structural response for implementation in practice.

Work-induced changes in feelings of mastery.

PubMed

Bradley, Graham L

2010-01-01

Past theory and research indicate that conditions of work can have lasting effects on job incumbents. R. A. Karasek and T. Theorell (1990), for example, proposed that workers' feelings of mastery increase with levels of job demands and job control, and that these effects are mediated by the process of active learning. To test these propositions, 657 school teachers completed scales assessing job demands, control, active learning, and mastery on 2 occasions, 8 months apart. As hypothesized, job control predicted change in mastery, an effect that was mediated by active learning. Job demands had a weaker effect on change in mastery. The demands-mastery relationship was moderated by job control, so that under conditions of high control, but not low control, increasing job demands were associated with gains in mastery. The findings partially support R. A. Karasek and T. Theorell's (1990) predictions regarding the main, interactive, and mediated effects of job conditions on employee mastery.

A hypothesis on a role of oxytocin in the social mechanisms of speech and vocal learning.

PubMed

Theofanopoulou, Constantina; Boeckx, Cedric; Jarvis, Erich D

2017-08-30

Language acquisition in humans and song learning in songbirds naturally happen as a social learning experience, providing an excellent opportunity to reveal social motivation and reward mechanisms that boost sensorimotor learning. Our knowledge about the molecules and circuits that control these social mechanisms for vocal learning and language is limited. Here we propose a hypothesis of a role for oxytocin (OT) in the social motivation and evolution of vocal learning and language. Building upon existing evidence, we suggest specific neural pathways and mechanisms through which OT might modulate vocal learning circuits in specific developmental stages. © 2017 The Authors.

A hypothesis on a role of oxytocin in the social mechanisms of speech and vocal learning

PubMed Central

Jarvis, Erich D.

2017-01-01

Language acquisition in humans and song learning in songbirds naturally happen as a social learning experience, providing an excellent opportunity to reveal social motivation and reward mechanisms that boost sensorimotor learning. Our knowledge about the molecules and circuits that control these social mechanisms for vocal learning and language is limited. Here we propose a hypothesis of a role for oxytocin (OT) in the social motivation and evolution of vocal learning and language. Building upon existing evidence, we suggest specific neural pathways and mechanisms through which OT might modulate vocal learning circuits in specific developmental stages. PMID:28835557

An Investigational Analysis of Problem Drinking among a Commuter College Population.

ERIC Educational Resources Information Center

Bebo, Joseph A.

2003-01-01

Previous literature has identified a number of variables that lead to substance abuse behavior. Studies that take a multifaceted, theoretical approach are limited. This study proposes the integration of differential association, social learning, social bonding, seduction and self-control theories. Proposes that a multifaceted approach contributes…

Adaptive iterative learning control of a class of nonlinear time-delay systems with unknown backlash-like hysteresis input and control direction.

PubMed

Wei, Jianming; Zhang, Youan; Sun, Meimei; Geng, Baoliang

2017-09-01

This paper presents an adaptive iterative learning control scheme for a class of nonlinear systems with unknown time-varying delays and control direction preceded by unknown nonlinear backlash-like hysteresis. Boundary layer function is introduced to construct an auxiliary error variable, which relaxes the identical initial condition assumption of iterative learning control. For the controller design, integral Lyapunov function candidate is used, which avoids the possible singularity problem by introducing hyperbolic tangent funciton. After compensating for uncertainties with time-varying delays by combining appropriate Lyapunov-Krasovskii function with Young's inequality, an adaptive iterative learning control scheme is designed through neural approximation technique and Nussbaum function method. On the basis of the hyperbolic tangent function's characteristics, the system output is proved to converge to a small neighborhood of the desired trajectory by constructing Lyapunov-like composite energy function (CEF) in two cases, while keeping all the closed-loop signals bounded. Finally, a simulation example is presented to verify the effectiveness of the proposed approach. Copyright © 2017 ISA. Published by Elsevier Ltd. All rights reserved.

Model-free learning on robot kinematic chains using a nested multi-agent topology

NASA Astrophysics Data System (ADS)

Karigiannis, John N.; Tzafestas, Costas S.

2016-11-01

This paper proposes a model-free learning scheme for the developmental acquisition of robot kinematic control and dexterous manipulation skills. The approach is based on a nested-hierarchical multi-agent architecture that intuitively encapsulates the topology of robot kinematic chains, where the activity of each independent degree-of-freedom (DOF) is finally mapped onto a distinct agent. Each one of those agents progressively evolves a local kinematic control strategy in a game-theoretic sense, that is, based on a partial (local) view of the whole system topology, which is incrementally updated through a recursive communication process according to the nested-hierarchical topology. Learning is thus approached not through demonstration and training but through an autonomous self-exploration process. A fuzzy reinforcement learning scheme is employed within each agent to enable efficient exploration in a continuous state-action domain. This paper constitutes in fact a proof of concept, demonstrating that global dexterous manipulation skills can indeed evolve through such a distributed iterative learning of local agent sensorimotor mappings. The main motivation behind the development of such an incremental multi-agent topology is to enhance system modularity, to facilitate extensibility to more complex problem domains and to improve robustness with respect to structural variations including unpredictable internal failures. These attributes of the proposed system are assessed in this paper through numerical experiments in different robot manipulation task scenarios, involving both single and multi-robot kinematic chains. The generalisation capacity of the learning scheme is experimentally assessed and robustness properties of the multi-agent system are also evaluated with respect to unpredictable variations in the kinematic topology. Furthermore, these numerical experiments demonstrate the scalability properties of the proposed nested-hierarchical architecture, where new agents can be recursively added in the hierarchy to encapsulate individual active DOFs. The results presented in this paper demonstrate the feasibility of such a distributed multi-agent control framework, showing that the solutions which emerge are plausible and near-optimal. Numerical efficiency and computational cost issues are also discussed.

GrDHP: a general utility function representation for dual heuristic dynamic programming.

PubMed

Ni, Zhen; He, Haibo; Zhao, Dongbin; Xu, Xin; Prokhorov, Danil V

2015-03-01

A general utility function representation is proposed to provide the required derivable and adjustable utility function for the dual heuristic dynamic programming (DHP) design. Goal representation DHP (GrDHP) is presented with a goal network being on top of the traditional DHP design. This goal network provides a general mapping between the system states and the derivatives of the utility function. With this proposed architecture, we can obtain the required derivatives of the utility function directly from the goal network. In addition, instead of a fixed predefined utility function in literature, we conduct an online learning process for the goal network so that the derivatives of the utility function can be adaptively tuned over time. We provide the control performance of both the proposed GrDHP and the traditional DHP approaches under the same environment and parameter settings. The statistical simulation results and the snapshot of the system variables are presented to demonstrate the improved learning and controlling performance. We also apply both approaches to a power system example to further demonstrate the control capabilities of the GrDHP approach.

Temperature based Restricted Boltzmann Machines

NASA Astrophysics Data System (ADS)

Li, Guoqi; Deng, Lei; Xu, Yi; Wen, Changyun; Wang, Wei; Pei, Jing; Shi, Luping

2016-01-01

Restricted Boltzmann machines (RBMs), which apply graphical models to learning probability distribution over a set of inputs, have attracted much attention recently since being proposed as building blocks of multi-layer learning systems called deep belief networks (DBNs). Note that temperature is a key factor of the Boltzmann distribution that RBMs originate from. However, none of existing schemes have considered the impact of temperature in the graphical model of DBNs. In this work, we propose temperature based restricted Boltzmann machines (TRBMs) which reveals that temperature is an essential parameter controlling the selectivity of the firing neurons in the hidden layers. We theoretically prove that the effect of temperature can be adjusted by setting the parameter of the sharpness of the logistic function in the proposed TRBMs. The performance of RBMs can be improved by adjusting the temperature parameter of TRBMs. This work provides a comprehensive insights into the deep belief networks and deep learning architectures from a physical point of view.

Birds, primates, and spoken language origins: behavioral phenotypes and neurobiological substrates

PubMed Central

Petkov, Christopher I.; Jarvis, Erich D.

2012-01-01

Vocal learners such as humans and songbirds can learn to produce elaborate patterns of structurally organized vocalizations, whereas many other vertebrates such as non-human primates and most other bird groups either cannot or do so to a very limited degree. To explain the similarities among humans and vocal-learning birds and the differences with other species, various theories have been proposed. One set of theories are motor theories, which underscore the role of the motor system as an evolutionary substrate for vocal production learning. For instance, the motor theory of speech and song perception proposes enhanced auditory perceptual learning of speech in humans and song in birds, which suggests a considerable level of neurobiological specialization. Another, a motor theory of vocal learning origin, proposes that the brain pathways that control the learning and production of song and speech were derived from adjacent motor brain pathways. Another set of theories are cognitive theories, which address the interface between cognition and the auditory-vocal domains to support language learning in humans. Here we critically review the behavioral and neurobiological evidence for parallels and differences between the so-called vocal learners and vocal non-learners in the context of motor and cognitive theories. In doing so, we note that behaviorally vocal-production learning abilities are more distributed than categorical, as are the auditory-learning abilities of animals. We propose testable hypotheses on the extent of the specializations and cross-species correspondences suggested by motor and cognitive theories. We believe that determining how spoken language evolved is likely to become clearer with concerted efforts in testing comparative data from many non-human animal species. PMID:22912615

Recurrent neural network control for LCC-resonant ultrasonic motor drive.

PubMed

Lin, F J; Wai, R J; Hong, C M

2000-01-01

A newly designed driving circuit for the traveling wave-type ultrasonic motor (USM), which consists of a push-pull DC-DC power converter and a two-phase voltage source inverter using one inductance and two capacitances (LCC) resonant technique, is presented in this study. Moreover, because the dynamic characteristics of the USM are difficult to obtain and the motor parameters are time varying, a recurrent neural network (RNN) controller is proposed to control the USM drive system. In the proposed controller, the dynamic backpropagation algorithm is adopted to train the RNN on-line using the proposed delta adaptation law. Furthermore, to guarantee the convergence of tracking error, analytical methods based on a discrete-type Lyapunov function are proposed to determine the varied learning rates for the training of the RNN. Finally, the effectiveness of the RNN-controlled USM drive system is demonstrated by some experimental results.

A self-learning rule base for command following in dynamical systems

NASA Technical Reports Server (NTRS)

Tsai, Wei K.; Lee, Hon-Mun; Parlos, Alexander

1992-01-01

In this paper, a self-learning Rule Base for command following in dynamical systems is presented. The learning is accomplished though reinforcement learning using an associative memory called SAM. The main advantage of SAM is that it is a function approximator with explicit storage of training samples. A learning algorithm patterned after the dynamic programming is proposed. Two artificially created, unstable dynamical systems are used for testing, and the Rule Base was used to generate a feedback control to improve the command following ability of the otherwise uncontrolled systems. The numerical results are very encouraging. The controlled systems exhibit a more stable behavior and a better capability to follow reference commands. The rules resulting from the reinforcement learning are explicitly stored and they can be modified or augmented by human experts. Due to overlapping storage scheme of SAM, the stored rules are similar to fuzzy rules.

Inclusion or Control? Commissioning and Contracting Services for People with Learning Disabilities

ERIC Educational Resources Information Center

Concannon, Liam

2006-01-01

The rise of new public management has seen the role of the social worker becoming increasingly administrative and less about face to face contact with service users. When commissioning managers seek to help people with learning disabilities plan their services, who actually makes the decisions? Direct payments are proposed as the answer for people…

Fixed Point Learning Based Intelligent Traffic Control System

NASA Astrophysics Data System (ADS)

Zongyao, Wang; Cong, Sui; Cheng, Shao

2017-10-01

Fixed point learning has become an important tool to analyse large scale distributed system such as urban traffic network. This paper presents a fixed point learning based intelligence traffic network control system. The system applies convergence property of fixed point theorem to optimize the traffic flow density. The intelligence traffic control system achieves maximum road resources usage by averaging traffic flow density among the traffic network. The intelligence traffic network control system is built based on decentralized structure and intelligence cooperation. No central control is needed to manage the system. The proposed system is simple, effective and feasible for practical use. The performance of the system is tested via theoretical proof and simulations. The results demonstrate that the system can effectively solve the traffic congestion problem and increase the vehicles average speed. It also proves that the system is flexible, reliable and feasible for practical use.

[CMACPAR an modified parallel neuro-controller for control processes].

PubMed

Ramos, E; Surós, R

1999-01-01

CMACPAR is a Parallel Neurocontroller oriented to real time systems as for example Control Processes. Its characteristics are mainly a fast learning algorithm, a reduced number of calculations, great generalization capacity, local learning and intrinsic parallelism. This type of neurocontroller is used in real time applications required by refineries, hydroelectric centers, factories, etc. In this work we present the analysis and the parallel implementation of a modified scheme of the Cerebellar Model CMAC for the n-dimensional space projection using a mean granularity parallel neurocontroller. The proposed memory management allows for a significant memory reduction in training time and required memory size.

An H(∞) control approach to robust learning of feedforward neural networks.

PubMed

Jing, Xingjian

2011-09-01

A novel H(∞) robust control approach is proposed in this study to deal with the learning problems of feedforward neural networks (FNNs). The analysis and design of a desired weight update law for the FNN is transformed into a robust controller design problem for a discrete dynamic system in terms of the estimation error. The drawbacks of some existing learning algorithms can therefore be revealed, especially for the case that the output data is fast changing with respect to the input or the output data is corrupted by noise. Based on this approach, the optimal learning parameters can be found by utilizing the linear matrix inequality (LMI) optimization techniques to achieve a predefined H(∞) "noise" attenuation level. Several existing BP-type algorithms are shown to be special cases of the new H(∞)-learning algorithm. Theoretical analysis and several examples are provided to show the advantages of the new method. Copyright © 2011 Elsevier Ltd. All rights reserved.

Reinforcement learning for a biped robot based on a CPG-actor-critic method.

PubMed

Nakamura, Yutaka; Mori, Takeshi; Sato, Masa-aki; Ishii, Shin

2007-08-01

Animals' rhythmic movements, such as locomotion, are considered to be controlled by neural circuits called central pattern generators (CPGs), which generate oscillatory signals. Motivated by this biological mechanism, studies have been conducted on the rhythmic movements controlled by CPG. As an autonomous learning framework for a CPG controller, we propose in this article a reinforcement learning method we call the "CPG-actor-critic" method. This method introduces a new architecture to the actor, and its training is roughly based on a stochastic policy gradient algorithm presented recently. We apply this method to an automatic acquisition problem of control for a biped robot. Computer simulations show that training of the CPG can be successfully performed by our method, thus allowing the biped robot to not only walk stably but also adapt to environmental changes.

Learning for intelligent mobile robots

NASA Astrophysics Data System (ADS)

Hall, Ernest L.; Liao, Xiaoqun; Alhaj Ali, Souma M.

2003-10-01

Unlike intelligent industrial robots which often work in a structured factory setting, intelligent mobile robots must often operate in an unstructured environment cluttered with obstacles and with many possible action paths. However, such machines have many potential applications in medicine, defense, industry and even the home that make their study important. Sensors such as vision are needed. However, in many applications some form of learning is also required. The purpose of this paper is to present a discussion of recent technical advances in learning for intelligent mobile robots. During the past 20 years, the use of intelligent industrial robots that are equipped not only with motion control systems but also with sensors such as cameras, laser scanners, or tactile sensors that permit adaptation to a changing environment has increased dramatically. However, relatively little has been done concerning learning. Adaptive and robust control permits one to achieve point to point and controlled path operation in a changing environment. This problem can be solved with a learning control. In the unstructured environment, the terrain and consequently the load on the robot"s motors are constantly changing. Learning the parameters of a proportional, integral and derivative controller (PID) and artificial neural network provides an adaptive and robust control. Learning may also be used for path following. Simulations that include learning may be conducted to see if a robot can learn its way through a cluttered array of obstacles. If a situation is performed repetitively, then learning can also be used in the actual application. To reach an even higher degree of autonomous operation, a new level of learning is required. Recently learning theories such as the adaptive critic have been proposed. In this type of learning a critic provides a grade to the controller of an action module such as a robot. The creative control process is used that is "beyond the adaptive critic." A mathematical model of the creative control process is presented that illustrates the use for mobile robots. Examples from a variety of intelligent mobile robot applications are also presented. The significance of this work is in providing a greater understanding of the applications of learning to mobile robots that could lead to many applications.

A Development of Game-Based Learning Environment to Activate Interaction among Learners

NASA Astrophysics Data System (ADS)

Takaoka, Ryo; Shimokawa, Masayuki; Okamoto, Toshio

Many studies and systems that incorporate elements such as “pleasure” and “fun” in the game to improve a learner's motivation have been developed in the field of learning environments. However, few are the studies of situations where many learners gather at a single computer and participate in a game-based learning environment (GBLE), and where the GBLE designs the learning process by controlling the interactions between learners such as competition, collaboration, and learning by teaching. Therefore, the purpose of this study is to propose a framework of educational control that induces and activates interaction between learners intentionally to create a learning opportunity that is based on the knowledge understanding model of each learner. In this paper, we explain the design philosophy and the framework of our GBLE called “Who becomes the king in the country of mathematics?” from a game viewpoint and describe the method of learning support control in the learning environment. In addition, we report the results of the learning experiment with our GBLE, which we carried out in a junior high school, and include some comments by a principal and a teacher. From the results of the experiment and some comments, we noticed that a game may play a significant role in weakening the learning relationship among students and creating new relationships in the world of the game. Furthermore, we discovered that learning support control of the GBLE has led to activation of the interaction between learners to some extent.

Translating theory into practice: integrating the affective and cognitive learning dimensions for effective instruction in engineering education

NASA Astrophysics Data System (ADS)

Alias, Maizam; Lashari, Tahira Anwar; Abidin Akasah, Zainal; Jahaya Kesot, Mohd.

2014-03-01

Learning in the cognitive domain is highly emphasised and has been widely investigated in engineering education. Lesser emphasis is placed on the affective dimension although the role of affects has been supported by research. The lack of understanding on learning theories and how they may be translated into classroom application of teaching and learning is one factor that contributes to this situation. This paper proposes a working framework for integrating the affective dimension of learning into engineering education that is expected to promote better learning within the cognitive domain. Four major learning theories namely behaviourism, cognitivism, socio-culturalism, and constructivism were analysed and how affects are postulated to influence cognition are identified. The affective domain constructs identified to be important are self-efficacy, attitude and locus of control. Based on the results of the analysis, a framework that integrates methodologies for achieving learning in the cognitive domain with the support of the affective dimension of learning is proposed. It is expected that integrated approach can be used as a guideline to engineering educators in designing effective and sustainable instructional material that would result in the effective engineers for future development.

Learning-based traffic signal control algorithms with neighborhood information sharing: An application for sustainable mobility

DOE Office of Scientific and Technical Information (OSTI.GOV)

Aziz, H. M. Abdul; Zhu, Feng; Ukkusuri, Satish V.

Here, this research applies R-Markov Average Reward Technique based reinforcement learning (RL) algorithm, namely RMART, for vehicular signal control problem leveraging information sharing among signal controllers in connected vehicle environment. We implemented the algorithm in a network of 18 signalized intersections and compare the performance of RMART with fixed, adaptive, and variants of the RL schemes. Results show significant improvement in system performance for RMART algorithm with information sharing over both traditional fixed signal timing plans and real time adaptive control schemes. Additionally, the comparison with reinforcement learning algorithms including Q learning and SARSA indicate that RMART performs better atmore » higher congestion levels. Further, a multi-reward structure is proposed that dynamically adjusts the reward function with varying congestion states at the intersection. Finally, the results from test networks show significant reduction in emissions (CO, CO 2, NO x, VOC, PM 10) when RL algorithms are implemented compared to fixed signal timings and adaptive schemes.« less

Simultaneous vibration control and energy harvesting using actor-critic based reinforcement learning

NASA Astrophysics Data System (ADS)

Loong, Cheng Ning; Chang, C. C.; Dimitrakopoulos, Elias G.

2018-03-01

Mitigating excessive vibration of civil engineering structures using various types of devices has been a conspicuous research topic in the past few decades. Some devices, such as electromagnetic transducers, which have a capability of exerting control forces while simultaneously harvesting energy, have been proposed recently. These devices make possible a self-regenerative system that can semi-actively mitigate structural vibration without the need of external energy. Integrating mechanical, electrical components, and control algorithms, these devices open up a new research domain that needs to be addressed. In this study, the feasibility of using an actor-critic based reinforcement learning control algorithm for simultaneous vibration control and energy harvesting for a civil engineering structure is investigated. The actor-critic based reinforcement learning control algorithm is a real-time, model-free adaptive technique that can adjust the controller parameters based on observations and reward signals without knowing the system characteristics. It is suitable for the control of a partially known nonlinear system with uncertain parameters. The feasibility of implementing this algorithm on a building structure equipped with an electromagnetic damper will be investigated in this study. Issues related to the modelling of learning algorithm, initialization and convergence will be presented and discussed.

Learning-based adaptive prescribed performance control of postcapture space robot-target combination without inertia identifications

NASA Astrophysics Data System (ADS)

Wei, Caisheng; Luo, Jianjun; Dai, Honghua; Bian, Zilin; Yuan, Jianping

2018-05-01

In this paper, a novel learning-based adaptive attitude takeover control method is investigated for the postcapture space robot-target combination with guaranteed prescribed performance in the presence of unknown inertial properties and external disturbance. First, a new static prescribed performance controller is developed to guarantee that all the involved attitude tracking errors are uniformly ultimately bounded by quantitatively characterizing the transient and steady-state performance of the combination. Then, a learning-based supplementary adaptive strategy based on adaptive dynamic programming is introduced to improve the tracking performance of static controller in terms of robustness and adaptiveness only utilizing the input/output data of the combination. Compared with the existing works, the prominent advantage is that the unknown inertial properties are not required to identify in the development of learning-based adaptive control law, which dramatically decreases the complexity and difficulty of the relevant controller design. Moreover, the transient and steady-state performance is guaranteed a priori by designer-specialized performance functions without resorting to repeated regulations of the controller parameters. Finally, the three groups of illustrative examples are employed to verify the effectiveness of the proposed control method.

Scaling Up Graph-Based Semisupervised Learning via Prototype Vector Machines

PubMed Central

Zhang, Kai; Lan, Liang; Kwok, James T.; Vucetic, Slobodan; Parvin, Bahram

2014-01-01

When the amount of labeled data are limited, semi-supervised learning can improve the learner's performance by also using the often easily available unlabeled data. In particular, a popular approach requires the learned function to be smooth on the underlying data manifold. By approximating this manifold as a weighted graph, such graph-based techniques can often achieve state-of-the-art performance. However, their high time and space complexities make them less attractive on large data sets. In this paper, we propose to scale up graph-based semisupervised learning using a set of sparse prototypes derived from the data. These prototypes serve as a small set of data representatives, which can be used to approximate the graph-based regularizer and to control model complexity. Consequently, both training and testing become much more efficient. Moreover, when the Gaussian kernel is used to define the graph affinity, a simple and principled method to select the prototypes can be obtained. Experiments on a number of real-world data sets demonstrate encouraging performance and scaling properties of the proposed approach. It also compares favorably with models learned via ℓ1-regularization at the same level of model sparsity. These results demonstrate the efficacy of the proposed approach in producing highly parsimonious and accurate models for semisupervised learning. PMID:25720002

Data-Driven H∞ Control for Nonlinear Distributed Parameter Systems.

PubMed

Luo, Biao; Huang, Tingwen; Wu, Huai-Ning; Yang, Xiong

2015-11-01

The data-driven H∞ control problem of nonlinear distributed parameter systems is considered in this paper. An off-policy learning method is developed to learn the H∞ control policy from real system data rather than the mathematical model. First, Karhunen-Loève decomposition is used to compute the empirical eigenfunctions, which are then employed to derive a reduced-order model (ROM) of slow subsystem based on the singular perturbation theory. The H∞ control problem is reformulated based on the ROM, which can be transformed to solve the Hamilton-Jacobi-Isaacs (HJI) equation, theoretically. To learn the solution of the HJI equation from real system data, a data-driven off-policy learning approach is proposed based on the simultaneous policy update algorithm and its convergence is proved. For implementation purpose, a neural network (NN)- based action-critic structure is developed, where a critic NN and two action NNs are employed to approximate the value function, control, and disturbance policies, respectively. Subsequently, a least-square NN weight-tuning rule is derived with the method of weighted residuals. Finally, the developed data-driven off-policy learning approach is applied to a nonlinear diffusion-reaction process, and the obtained results demonstrate its effectiveness.

77 FR 4287 - Notice of Proposed Information Collection Requests

Federal Register 2010, 2011, 2012, 2013, 2014

2012-01-27

... other will not have experienced the feature (``control''). Average test scores of the two groups will... Promising Features of Teacher Preparation Programs; Phase 1--Recruitment. OMB Control Number: 1850-NEW... baseline student achievement test for an experimental study of the effect on student learning of teachers...

Reinforcement learning state estimator.

PubMed

Morimoto, Jun; Doya, Kenji

2007-03-01

In this study, we propose a novel use of reinforcement learning for estimating hidden variables and parameters of nonlinear dynamical systems. A critical issue in hidden-state estimation is that we cannot directly observe estimation errors. However, by defining errors of observable variables as a delayed penalty, we can apply a reinforcement learning frame-work to state estimation problems. Specifically, we derive a method to construct a nonlinear state estimator by finding an appropriate feedback input gain using the policy gradient method. We tested the proposed method on single pendulum dynamics and show that the joint angle variable could be successfully estimated by observing only the angular velocity, and vice versa. In addition, we show that we could acquire a state estimator for the pendulum swing-up task in which a swing-up controller is also acquired by reinforcement learning simultaneously. Furthermore, we demonstrate that it is possible to estimate the dynamics of the pendulum itself while the hidden variables are estimated in the pendulum swing-up task. Application of the proposed method to a two-linked biped model is also presented.

Model-Free Primitive-Based Iterative Learning Control Approach to Trajectory Tracking of MIMO Systems With Experimental Validation.

PubMed

Radac, Mircea-Bogdan; Precup, Radu-Emil; Petriu, Emil M

2015-11-01

This paper proposes a novel model-free trajectory tracking of multiple-input multiple-output (MIMO) systems by the combination of iterative learning control (ILC) and primitives. The optimal trajectory tracking solution is obtained in terms of previously learned solutions to simple tasks called primitives. The library of primitives that are stored in memory consists of pairs of reference input/controlled output signals. The reference input primitives are optimized in a model-free ILC framework without using knowledge of the controlled process. The guaranteed convergence of the learning scheme is built upon a model-free virtual reference feedback tuning design of the feedback decoupling controller. Each new complex trajectory to be tracked is decomposed into the output primitives regarded as basis functions. The optimal reference input for the control system to track the desired trajectory is next recomposed from the reference input primitives. This is advantageous because the optimal reference input is computed straightforward without the need to learn from repeated executions of the tracking task. In addition, the optimization problem specific to trajectory tracking of square MIMO systems is decomposed in a set of optimization problems assigned to each separate single-input single-output control channel that ensures a convenient model-free decoupling. The new model-free primitive-based ILC approach is capable of planning, reasoning, and learning. A case study dealing with the model-free control tuning for a nonlinear aerodynamic system is included to validate the new approach. The experimental results are given.

A self-agency bias in preschoolers' causal inferences

PubMed Central

Kushnir, Tamar; Wellman, Henry M.; Gelman, Susan A.

2013-01-01

Preschoolers' causal learning from intentional actions – causal interventions – is subject to a self-agency bias. We propose that this bias is evidence-based; it is responsive to causal uncertainty. In the current studies, two causes (one child-controlled, one experimenter-controlled) were associated with one or two effects, first independently, then simultaneously. When initial independent effects were probabilistic, and thus subsequent simultaneous actions were causally ambiguous, children showed a self-agency bias. Children showed no bias when initial effects were deterministic. Further controls establish that children's self-agency bias is not a wholesale preference but rather is influenced by uncertainty in causal evidence. These results demonstrate that children's own experience of action influences their causal learning, and suggest possible benefits in uncertain and ambiguous everyday learning contexts. PMID:19271843

Dynamic neural networks based on-line identification and control of high performance motor drives

NASA Technical Reports Server (NTRS)

Rubaai, Ahmed; Kotaru, Raj

1995-01-01

In the automated and high-tech industries of the future, there wil be a need for high performance motor drives both in the low-power range and in the high-power range. To meet very straight demands of tracking and regulation in the two quadrants of operation, advanced control technologies are of a considerable interest and need to be developed. In response a dynamics learning control architecture is developed with simultaneous on-line identification and control. the feature of the proposed approach, to efficiently combine the dual task of system identification (learning) and adaptive control of nonlinear motor drives into a single operation is presented. This approach, therefore, not only adapts to uncertainties of the dynamic parameters of the motor drives but also learns about their inherent nonlinearities. In fact, most of the neural networks based adaptive control approaches in use have an identification phase entirely separate from the control phase. Because these approaches separate the identification and control modes, it is not possible to cope with dynamic changes in a controlled process. Extensive simulation studies have been conducted and good performance was observed. The robustness characteristics of neuro-controllers to perform efficiently in a noisy environment is also demonstrated. With this initial success, the principal investigator believes that the proposed approach with the suggested neural structure can be used successfully for the control of high performance motor drives. Two identification and control topologies based on the model reference adaptive control technique are used in this present analysis. No prior knowledge of load dynamics is assumed in either topology while the second topology also assumes no knowledge of the motor parameters.

Inclusion of Immersive Virtual Learning Environments and Visual Control Systems to Support the Learning of Students with Asperger Syndrome

ERIC Educational Resources Information Center

Lorenzo, Gonzalo; Pomares, Jorge; Lledo, Asuncion

2013-01-01

This paper presents the use of immersive virtual reality systems in the educational intervention with Asperger students. The starting points of this study are features of these students' cognitive style that requires an explicit teaching style supported by visual aids and highly structured environments. The proposed immersive virtual reality…

Implicit and explicit motor sequence learning in children born very preterm.

PubMed

Jongbloed-Pereboom, Marjolein; Janssen, Anjo J W M; Steiner, K; Steenbergen, Bert; Nijhuis-van der Sanden, Maria W G

2017-01-01

Motor skills can be learned explicitly (dependent on working memory (WM)) or implicitly (relatively independent of WM). Children born very preterm (VPT) often have working memory deficits. Explicit learning may be compromised in these children. This study investigated implicit and explicit motor learning and the role of working memory in VPT children and controls. Three groups (6-9 years) participated: 20 VPT children with motor problems, 20 VPT children without motor problems, and 20 controls. A nine button sequence was learned implicitly (pressing the lighted button as quickly as possible) and explicitly (discovering the sequence via trial-and-error). Children learned implicitly and explicitly, evidenced by decreased movement duration of the sequence over time. In the explicit condition, children also reduced the number of errors over time. Controls made more errors than VPT children without motor problems. Visual WM had positive effects on both explicit and implicit performance. VPT birth and low motor proficiency did not negatively affect implicit or explicit learning. Visual WM was positively related to both implicit and explicit performance, but did not influence learning curves. These findings question the theoretical difference between implicit and explicit learning and the proposed role of visual WM therein. Copyright © 2016 Elsevier Ltd. All rights reserved.

Rewiring the Brain: Potential Role of the Premotor Cortex in Motor Control, Learning, and Recovery of Function Following Brain Injury

PubMed Central

Kantak, Shailesh S.; Stinear, James W.; Buch, Ethan R.; Cohen, Leonardo G.

2016-01-01

The brain is a plastic organ with a capability to reorganize in response to behavior and/or injury. Following injury to the motor cortex or emergent corticospinal pathways, recovery of function depends on the capacity of surviving anatomical resources to recover and repair in response to task-specific training. One such area implicated in poststroke reorganization to promote recovery of upper extremity recovery is the premotor cortex (PMC). This study reviews the role of distinct subdivisions of PMC: dorsal (PMd) and ventral (PMv) premotor cortices as critical anatomical and physiological nodes within the neural networks for the control and learning of goal-oriented reach and grasp actions in healthy individuals and individuals with stroke. Based on evidence emerging from studies of intrinsic and extrinsic connectivity, transcranial magnetic stimulation, functional neuroimaging, and experimental studies in animals and humans, the authors propose 2 distinct patterns of reorganization that differentially engage ipsilesional and contralesional PMC. Research directions that may offer further insights into the role of PMC in motor control, learning, and poststroke recovery are also proposed. This research may facilitate neuroplasticity for maximal recovery of function following brain injury. PMID:21926382

Robust reinforcement learning.

PubMed

Morimoto, Jun; Doya, Kenji

2005-02-01

This letter proposes a new reinforcement learning (RL) paradigm that explicitly takes into account input disturbance as well as modeling errors. The use of environmental models in RL is quite popular for both offline learning using simulations and for online action planning. However, the difference between the model and the real environment can lead to unpredictable, and often unwanted, results. Based on the theory of H(infinity) control, we consider a differential game in which a "disturbing" agent tries to make the worst possible disturbance while a "control" agent tries to make the best control input. The problem is formulated as finding a min-max solution of a value function that takes into account the amount of the reward and the norm of the disturbance. We derive online learning algorithms for estimating the value function and for calculating the worst disturbance and the best control in reference to the value function. We tested the paradigm, which we call robust reinforcement learning (RRL), on the control task of an inverted pendulum. In the linear domain, the policy and the value function learned by online algorithms coincided with those derived analytically by the linear H(infinity) control theory. For a fully nonlinear swing-up task, RRL achieved robust performance with changes in the pendulum weight and friction, while a standard reinforcement learning algorithm could not deal with these changes. We also applied RRL to the cart-pole swing-up task, and a robust swing-up policy was acquired.

Bio-inspired spiking neural network for nonlinear systems control.

PubMed

Pérez, Javier; Cabrera, Juan A; Castillo, Juan J; Velasco, Juan M

2018-08-01

Spiking neural networks (SNN) are the third generation of artificial neural networks. SNN are the closest approximation to biological neural networks. SNNs make use of temporal spike trains to command inputs and outputs, allowing a faster and more complex computation. As demonstrated by biological organisms, they are a potentially good approach to designing controllers for highly nonlinear dynamic systems in which the performance of controllers developed by conventional techniques is not satisfactory or difficult to implement. SNN-based controllers exploit their ability for online learning and self-adaptation to evolve when transferred from simulations to the real world. SNN's inherent binary and temporary way of information codification facilitates their hardware implementation compared to analog neurons. Biological neural networks often require a lower number of neurons compared to other controllers based on artificial neural networks. In this work, these neuronal systems are imitated to perform the control of non-linear dynamic systems. For this purpose, a control structure based on spiking neural networks has been designed. Particular attention has been paid to optimizing the structure and size of the neural network. The proposed structure is able to control dynamic systems with a reduced number of neurons and connections. A supervised learning process using evolutionary algorithms has been carried out to perform controller training. The efficiency of the proposed network has been verified in two examples of dynamic systems control. Simulations show that the proposed control based on SNN exhibits superior performance compared to other approaches based on Neural Networks and SNNs. Copyright © 2018 Elsevier Ltd. All rights reserved.

Motivation of extended behaviors by anterior cingulate cortex.

PubMed

Holroyd, Clay B; Yeung, Nick

2012-02-01

Intense research interest over the past decade has yielded diverse and often discrepant theories about the function of anterior cingulate cortex (ACC). In particular, a dichotomy has emerged between neuropsychological theories suggesting a primary role for ACC in motivating or 'energizing' behavior, and neuroimaging-inspired theories emphasizing its contribution to cognitive control and reinforcement learning. To reconcile these views, we propose that ACC supports the selection and maintenance of 'options' - extended, context-specific sequences of behavior directed toward particular goals - that are learned through a process of hierarchical reinforcement learning. This theory accounts for ACC activity in relation to learning and control while simultaneously explaining the effects of ACC damage as disrupting the motivational context supporting the production of goal-directed action sequences. Copyright © 2011 Elsevier Ltd. All rights reserved.

Developing Humanoid Robots for Real-World Environments

NASA Technical Reports Server (NTRS)

Stoica, Adrian; Kuhlman, Michael; Assad, Chris; Keymeulen, Didier

2008-01-01

Humanoids are steadily improving in appearance and functionality demonstrated in controlled environments. To address the challenges of operation in the real-world, researchers have proposed the use of brain-inspired architectures for robot control, and the use of robot learning techniques that enable the robot to acquire and tune skills and behaviours. In the first part of the paper we introduce new concepts and results in these two areas. First, we present a cerebellum-inspired model that demonstrated efficiency in the sensory-motor control of anthropomorphic arms, and in gait control of dynamic walkers. Then, we present a set of new ideas related to robot learning, emphasizing the importance of developing teaching techniques that support learning. In the second part of the paper we propose the use in robotics of the iterative and incremental development methodologies, in the context of practical task-oriented applications. These methodologies promise to rapidly reach system-level integration, and to early identify system-level weaknesses to focus on. We apply this methodology in a task targeting the automated assembly of a modular structure using HOAP-2. We confirm this approach led to rapid development of a end-to-end capability, and offered guidance on which technologies to focus on for gradual improvement of a complete functional system. It is believed that providing Grand Challenge type milestones in practical task-oriented applications accelerates development. As a meaningful target in short-mid term we propose the 'IKEA Challenge', aimed at the demonstration of autonomous assembly of various pieces of furniture, from the box, following included written/drawn instructions.

Aggregative Learning Method and Its Application for Communication Quality Evaluation

NASA Astrophysics Data System (ADS)

Akhmetov, Dauren F.; Kotaki, Minoru

2007-12-01

In this paper, so-called Aggregative Learning Method (ALM) is proposed to improve and simplify the learning and classification abilities of different data processing systems. It provides a universal basis for design and analysis of mathematical models of wide class. A procedure was elaborated for time series model reconstruction and analysis for linear and nonlinear cases. Data approximation accuracy (during learning phase) and data classification quality (during recall phase) are estimated from introduced statistic parameters. The validity and efficiency of the proposed approach have been demonstrated through its application for monitoring of wireless communication quality, namely, for Fixed Wireless Access (FWA) system. Low memory and computation resources were shown to be needed for the procedure realization, especially for data classification (recall) stage. Characterized with high computational efficiency and simple decision making procedure, the derived approaches can be useful for simple and reliable real-time surveillance and control system design.

Novel bio-inspired smart control for hazard mitigation of civil structures

NASA Astrophysics Data System (ADS)

Kim, Yeesock; Kim, Changwon; Langari, Reza

2010-11-01

In this paper, a new bio-inspired controller is proposed for vibration mitigation of smart structures subjected to ground disturbances (i.e. earthquakes). The control system is developed through the integration of a brain emotional learning (BEL) algorithm with a proportional-integral-derivative (PID) controller and a semiactive inversion (Inv) algorithm. The BEL algorithm is based on the neurologically inspired computational model of the amygdala and the orbitofrontal cortex. To demonstrate the effectiveness of the proposed hybrid BEL-PID-Inv control algorithm, a seismically excited building structure equipped with a magnetorheological (MR) damper is investigated. The performance of the proposed hybrid BEL-PID-Inv control algorithm is compared with that of passive, PID, linear quadratic Gaussian (LQG), and BEL control systems. In the simulation, the robustness of the hybrid BEL-PID-Inv control algorithm in the presence of modeling uncertainties as well as external disturbances is investigated. It is shown that the proposed hybrid BEL-PID-Inv control algorithm is effective in improving the dynamic responses of seismically excited building structure-MR damper systems.

A top-down approach in control engineering third-level teaching: The case of hydrogen-generation

NASA Astrophysics Data System (ADS)

Setiawan, Eko; Habibi, M. Afnan; Fall, Cheikh; Hodaka, Ichijo

2017-09-01

This paper presents a top-down approach in control engineering third-level teaching. The paper shows the control engineering solution for the issue of practical implementation in order to motivate students. The proposed strategy only focuses on one technique of control engineering to lead student correctly. The proposed teaching steps are 1) defining the problem, 2) listing of acquired knowledge or required skill, 3) selecting of one control engineering technique, 4) arrangement the order of teaching: problem introduction, implementation of control engineering technique, explanation of system block diagram, model derivation, controller design, and 5) enrichment knowledge by the other control techniques. The approach presented highlights hardware implementation and the use of software simulation as a self-learning tool for students.

What No Child Left Behind Leaves Behind: The Roles of IQ and Self-Control in Predicting Standardized Achievement Test Scores and Report Card Grades

PubMed Central

Duckworth, Angela L.; Quinn, Patrick D.; Tsukayama, Eli

2013-01-01

The increasing prominence of standardized testing to assess student learning motivated the current investigation. We propose that standardized achievement test scores assess competencies determined more by intelligence than by self-control, whereas report card grades assess competencies determined more by self-control than by intelligence. In particular, we suggest that intelligence helps students learn and solve problems independent of formal instruction, whereas self-control helps students study, complete homework, and behave positively in the classroom. Two longitudinal, prospective studies of middle school students support predictions from this model. In both samples, IQ predicted changes in standardized achievement test scores over time better than did self-control, whereas self-control predicted changes in report card grades over time better than did IQ. As expected, the effect of self-control on changes in report card grades was mediated in Study 2 by teacher ratings of homework completion and classroom conduct. In a third study, ratings of middle school teachers about the content and purpose of standardized achievement tests and report card grades were consistent with the proposed model. Implications for pedagogy and public policy are discussed. PMID:24072936

Nonparametric Online Learning Control for Soft Continuum Robot: An Enabling Technique for Effective Endoscopic Navigation

PubMed Central

Lee, Kit-Hang; Fu, Denny K.C.; Leong, Martin C.W.; Chow, Marco; Fu, Hing-Choi; Althoefer, Kaspar; Sze, Kam Yim; Yeung, Chung-Kwong

2017-01-01

Abstract Bioinspired robotic structures comprising soft actuation units have attracted increasing research interest. Taking advantage of its inherent compliance, soft robots can assure safe interaction with external environments, provided that precise and effective manipulation could be achieved. Endoscopy is a typical application. However, previous model-based control approaches often require simplified geometric assumptions on the soft manipulator, but which could be very inaccurate in the presence of unmodeled external interaction forces. In this study, we propose a generic control framework based on nonparametric and online, as well as local, training to learn the inverse model directly, without prior knowledge of the robot's structural parameters. Detailed experimental evaluation was conducted on a soft robot prototype with control redundancy, performing trajectory tracking in dynamically constrained environments. Advanced element formulation of finite element analysis is employed to initialize the control policy, hence eliminating the need for random exploration in the robot's workspace. The proposed control framework enabled a soft fluid-driven continuum robot to follow a 3D trajectory precisely, even under dynamic external disturbance. Such enhanced control accuracy and adaptability would facilitate effective endoscopic navigation in complex and changing environments. PMID:29251567

Nonparametric Online Learning Control for Soft Continuum Robot: An Enabling Technique for Effective Endoscopic Navigation.

PubMed

Lee, Kit-Hang; Fu, Denny K C; Leong, Martin C W; Chow, Marco; Fu, Hing-Choi; Althoefer, Kaspar; Sze, Kam Yim; Yeung, Chung-Kwong; Kwok, Ka-Wai

2017-12-01

Bioinspired robotic structures comprising soft actuation units have attracted increasing research interest. Taking advantage of its inherent compliance, soft robots can assure safe interaction with external environments, provided that precise and effective manipulation could be achieved. Endoscopy is a typical application. However, previous model-based control approaches often require simplified geometric assumptions on the soft manipulator, but which could be very inaccurate in the presence of unmodeled external interaction forces. In this study, we propose a generic control framework based on nonparametric and online, as well as local, training to learn the inverse model directly, without prior knowledge of the robot's structural parameters. Detailed experimental evaluation was conducted on a soft robot prototype with control redundancy, performing trajectory tracking in dynamically constrained environments. Advanced element formulation of finite element analysis is employed to initialize the control policy, hence eliminating the need for random exploration in the robot's workspace. The proposed control framework enabled a soft fluid-driven continuum robot to follow a 3D trajectory precisely, even under dynamic external disturbance. Such enhanced control accuracy and adaptability would facilitate effective endoscopic navigation in complex and changing environments.

Designing the Undesignable: Social Software and Control

ERIC Educational Resources Information Center

Dron, Jon

2007-01-01

Social software, such as blogs, wikis, tagging systems and collaborative filters, treats the group as a first-class object within the system. Drawing from theories of transactional distance and control, this paper proposes a model of e-learning that extends traditional concepts of learner-teacher-content interactions to include these emergent…

Mild cognitive impairment affects motor control and skill learning.

PubMed

Wu, Qiaofeng; Chan, John S Y; Yan, Jin H

2016-02-01

Mild cognitive impairment (MCI) is a transitional phase between normal cognitive aging and dementia. As the world population is aging rapidly, more MCI patients will be identified, posing significant problems to society. Normal aging is associated with cognitive and motor decline, and MCI brings additional impairments. Compared to healthy older adults, MCI patients show poorer motor control in a variety of tasks. Efficient motor control and skill learning are essential for occupational and leisure purposes; degradation of motor behaviors in MCI patients often adversely affects their health and quality of life. In this article, we first define MCI and describe its pathology and neural correlates. After this, we review cognitive changes and motor control and skill learning in normal aging. This section is followed by a discussion of MCI-related degradation of motor behaviors. Finally, we propose that multicomponent interventions targeting both cognitive and motor domains can improve MCI patients' motor functions. Future research directions are also raised.

The active learning hypothesis of the job-demand-control model: an experimental examination.

PubMed

Häusser, Jan Alexander; Schulz-Hardt, Stefan; Mojzisch, Andreas

2014-01-01

The active learning hypothesis of the job-demand-control model [Karasek, R. A. 1979. "Job Demands, Job Decision Latitude, and Mental Strain: Implications for Job Redesign." Administration Science Quarterly 24: 285-307] proposes positive effects of high job demands and high job control on performance. We conducted a 2 (demands: high vs. low) × 2 (control: high vs. low) experimental office workplace simulation to examine this hypothesis. Since performance during a work simulation is confounded by the boundaries of the demands and control manipulations (e.g. time limits), we used a post-test, in which participants continued working at their task, but without any manipulation of demands and control. This post-test allowed for examining active learning (transfer) effects in an unconfounded fashion. Our results revealed that high demands had a positive effect on quantitative performance, without affecting task accuracy. In contrast, high control resulted in a speed-accuracy tradeoff, that is participants in the high control conditions worked slower but with greater accuracy than participants in the low control conditions.

Adaptive Event-Triggered Control Based on Heuristic Dynamic Programming for Nonlinear Discrete-Time Systems.

PubMed

Dong, Lu; Zhong, Xiangnan; Sun, Changyin; He, Haibo

2017-07-01

This paper presents the design of a novel adaptive event-triggered control method based on the heuristic dynamic programming (HDP) technique for nonlinear discrete-time systems with unknown system dynamics. In the proposed method, the control law is only updated when the event-triggered condition is violated. Compared with the periodic updates in the traditional adaptive dynamic programming (ADP) control, the proposed method can reduce the computation and transmission cost. An actor-critic framework is used to learn the optimal event-triggered control law and the value function. Furthermore, a model network is designed to estimate the system state vector. The main contribution of this paper is to design a new trigger threshold for discrete-time systems. A detailed Lyapunov stability analysis shows that our proposed event-triggered controller can asymptotically stabilize the discrete-time systems. Finally, we test our method on two different discrete-time systems, and the simulation results are included.

Indirect iterative learning control for a discrete visual servo without a camera-robot model.

PubMed

Jiang, Ping; Bamforth, Leon C A; Feng, Zuren; Baruch, John E F; Chen, YangQuan

2007-08-01

This paper presents a discrete learning controller for vision-guided robot trajectory imitation with no prior knowledge of the camera-robot model. A teacher demonstrates a desired movement in front of a camera, and then, the robot is tasked to replay it by repetitive tracking. The imitation procedure is considered as a discrete tracking control problem in the image plane, with an unknown and time-varying image Jacobian matrix. Instead of updating the control signal directly, as is usually done in iterative learning control (ILC), a series of neural networks are used to approximate the unknown Jacobian matrix around every sample point in the demonstrated trajectory, and the time-varying weights of local neural networks are identified through repetitive tracking, i.e., indirect ILC. This makes repetitive segmented training possible, and a segmented training strategy is presented to retain the training trajectories solely within the effective region for neural network approximation. However, a singularity problem may occur if an unmodified neural-network-based Jacobian estimation is used to calculate the robot end-effector velocity. A new weight modification algorithm is proposed which ensures invertibility of the estimation, thus circumventing the problem. Stability is further discussed, and the relationship between the approximation capability of the neural network and the tracking accuracy is obtained. Simulations and experiments are carried out to illustrate the validity of the proposed controller for trajectory imitation of robot manipulators with unknown time-varying Jacobian matrices.

What FDA Learned About Dark Chocolate and Milk Allergies

MedlinePlus

... the issues identified in the study. Further, allergen contamination is included in the preventive and risk-based controls mandated by the FDA Food Safety Modernization Act (FSMA). Under the proposed Preventive ...

Haptic-Based Perception-Empathy Biofeedback Enhances Postural Motor Learning During High-Cognitive Load Task in Healthy Older Adults

PubMed Central

Yasuda, Kazuhiro; Saichi, Kenta; Iwata, Hiroyasu

2018-01-01

Falls and fall-induced injuries are major global public health problems, and sensory input impairment in older adults results in significant limitations in feedback-type postural control. A haptic-based biofeedback (BF) system can be used for augmenting somatosensory input in older adults, and the application of this BF system can increase the objectivity of the feedback and encourage comparison with that provided by a trainer. Nevertheless, an optimal BF system that focuses on interpersonal feedback for balance training in older adults has not been proposed. Thus, we proposed a haptic-based perception-empathy BF system that provides information regarding the older adult's center-of-foot pressure pattern to the trainee and trainer for refining the motor learning effect. The first objective of this study was to examine the effect of this balance training regimen in healthy older adults performing a postural learning task. Second, this study aimed to determine whether BF training required high cognitive load to clarify its practicability in real-life settings. Twenty older adults were assigned to two groups: BF and control groups. Participants in both groups tried balance training in the single-leg stance while performing a cognitive task (i.e., serial subtraction task). Retention was tested 24 h later. Testing comprised balance performance measures (i.e., 95% confidence ellipse area and mean velocity of sway) and dual-task performance (number of responses and correct answers). Measurements of postural control using a force plate revealed that the stability of the single-leg stance was significantly lower in the BF group than in the control group during the balance task. The BF group retained the improvement in the 95% confidence ellipse area 24 h after the retention test. Results of dual-task performance during the balance task were not different between the two groups. These results confirmed the potential benefit of the proposed balance training regimen in designing successful motor learning programs for preventing falls in older adults. PMID:29868597

Haptic-Based Perception-Empathy Biofeedback Enhances Postural Motor Learning During High-Cognitive Load Task in Healthy Older Adults.

PubMed

Yasuda, Kazuhiro; Saichi, Kenta; Iwata, Hiroyasu

2018-01-01

Falls and fall-induced injuries are major global public health problems, and sensory input impairment in older adults results in significant limitations in feedback-type postural control. A haptic-based biofeedback (BF) system can be used for augmenting somatosensory input in older adults, and the application of this BF system can increase the objectivity of the feedback and encourage comparison with that provided by a trainer. Nevertheless, an optimal BF system that focuses on interpersonal feedback for balance training in older adults has not been proposed. Thus, we proposed a haptic-based perception-empathy BF system that provides information regarding the older adult's center-of-foot pressure pattern to the trainee and trainer for refining the motor learning effect. The first objective of this study was to examine the effect of this balance training regimen in healthy older adults performing a postural learning task. Second, this study aimed to determine whether BF training required high cognitive load to clarify its practicability in real-life settings. Twenty older adults were assigned to two groups: BF and control groups. Participants in both groups tried balance training in the single-leg stance while performing a cognitive task (i.e., serial subtraction task). Retention was tested 24 h later. Testing comprised balance performance measures (i.e., 95% confidence ellipse area and mean velocity of sway) and dual-task performance (number of responses and correct answers). Measurements of postural control using a force plate revealed that the stability of the single-leg stance was significantly lower in the BF group than in the control group during the balance task. The BF group retained the improvement in the 95% confidence ellipse area 24 h after the retention test. Results of dual-task performance during the balance task were not different between the two groups. These results confirmed the potential benefit of the proposed balance training regimen in designing successful motor learning programs for preventing falls in older adults.

Online Learning of Genetic Network Programming and its Application to Prisoner’s Dilemma Game

NASA Astrophysics Data System (ADS)

Mabu, Shingo; Hirasawa, Kotaro; Hu, Jinglu; Murata, Junichi

A new evolutionary model with the network structure named Genetic Network Programming (GNP) has been proposed recently. GNP, that is, an expansion of GA and GP, represents solutions as a network structure and evolves it by using “offline learning (selection, mutation, crossover)”. GNP can memorize the past action sequences in the network flow, so it can deal with Partially Observable Markov Decision Process (POMDP) well. In this paper, in order to improve the ability of GNP, Q learning (an off-policy TD control algorithm) that is one of the famous online methods is introduced for online learning of GNP. Q learning is suitable for GNP because (1) in reinforcement learning, the rewards an agent will get in the future can be estimated, (2) TD control doesn’t need much memory and can learn quickly, and (3) off-policy is suitable in order to search for an optimal solution independently of the policy. Finally, in the simulations, online learning of GNP is applied to a player for “Prisoner’s dilemma game” and its ability for online adaptation is confirmed.

Effects of arousal on cognitive control: empirical tests of the conflict-modulated Hebbian-learning hypothesis.

PubMed

Brown, Stephen B R E; van Steenbergen, Henk; Kedar, Tomer; Nieuwenhuis, Sander

2014-01-01

An increasing number of empirical phenomena that were previously interpreted as a result of cognitive control, turn out to reflect (in part) simple associative-learning effects. A prime example is the proportion congruency effect, the finding that interference effects (such as the Stroop effect) decrease as the proportion of incongruent stimuli increases. While this was previously regarded as strong evidence for a global conflict monitoring-cognitive control loop, recent evidence has shown that the proportion congruency effect is largely item-specific and hence must be due to associative learning. The goal of our research was to test a recent hypothesis about the mechanism underlying such associative-learning effects, the conflict-modulated Hebbian-learning hypothesis, which proposes that the effect of conflict on associative learning is mediated by phasic arousal responses. In Experiment 1, we examined in detail the relationship between the item-specific proportion congruency effect and an autonomic measure of phasic arousal: task-evoked pupillary responses. In Experiment 2, we used a task-irrelevant phasic arousal manipulation and examined the effect on item-specific learning of incongruent stimulus-response associations. The results provide little evidence for the conflict-modulated Hebbian-learning hypothesis, which requires additional empirical support to remain tenable.

Neural network based online simultaneous policy update algorithm for solving the HJI equation in nonlinear H∞ control.

PubMed

Wu, Huai-Ning; Luo, Biao

2012-12-01

It is well known that the nonlinear H∞ state feedback control problem relies on the solution of the Hamilton-Jacobi-Isaacs (HJI) equation, which is a nonlinear partial differential equation that has proven to be impossible to solve analytically. In this paper, a neural network (NN)-based online simultaneous policy update algorithm (SPUA) is developed to solve the HJI equation, in which knowledge of internal system dynamics is not required. First, we propose an online SPUA which can be viewed as a reinforcement learning technique for two players to learn their optimal actions in an unknown environment. The proposed online SPUA updates control and disturbance policies simultaneously; thus, only one iterative loop is needed. Second, the convergence of the online SPUA is established by proving that it is mathematically equivalent to Newton's method for finding a fixed point in a Banach space. Third, we develop an actor-critic structure for the implementation of the online SPUA, in which only one critic NN is needed for approximating the cost function, and a least-square method is given for estimating the NN weight parameters. Finally, simulation studies are provided to demonstrate the effectiveness of the proposed algorithm.

What "No Child Left Behind" Leaves behind: The Roles of IQ and Self-Control in Predicting Standardized Achievement Test Scores and Report Card Grades

ERIC Educational Resources Information Center

Duckworth, Angela L.; Quinn, Patrick D.; Tsukayama, Eli

2012-01-01

The increasing prominence of standardized testing to assess student learning motivated the current investigation. We propose that standardized achievement test scores assess competencies determined more by intelligence than by self-control, whereas report card grades assess competencies determined more by self-control than by intelligence. In…

Optimization Control of the Color-Coating Production Process for Model Uncertainty

PubMed Central

He, Dakuo; Wang, Zhengsong; Yang, Le; Mao, Zhizhong

2016-01-01

Optimized control of the color-coating production process (CCPP) aims at reducing production costs and improving economic efficiency while meeting quality requirements. However, because optimization control of the CCPP is hampered by model uncertainty, a strategy that considers model uncertainty is proposed. Previous work has introduced a mechanistic model of CCPP based on process analysis to simulate the actual production process and generate process data. The partial least squares method is then applied to develop predictive models of film thickness and economic efficiency. To manage the model uncertainty, the robust optimization approach is introduced to improve the feasibility of the optimized solution. Iterative learning control is then utilized to further refine the model uncertainty. The constrained film thickness is transformed into one of the tracked targets to overcome the drawback that traditional iterative learning control cannot address constraints. The goal setting of economic efficiency is updated continuously according to the film thickness setting until this reaches its desired value. Finally, fuzzy parameter adjustment is adopted to ensure that the economic efficiency and film thickness converge rapidly to their optimized values under the constraint conditions. The effectiveness of the proposed optimization control strategy is validated by simulation results. PMID:27247563

Optimization Control of the Color-Coating Production Process for Model Uncertainty.

PubMed

He, Dakuo; Wang, Zhengsong; Yang, Le; Mao, Zhizhong

2016-01-01

Optimized control of the color-coating production process (CCPP) aims at reducing production costs and improving economic efficiency while meeting quality requirements. However, because optimization control of the CCPP is hampered by model uncertainty, a strategy that considers model uncertainty is proposed. Previous work has introduced a mechanistic model of CCPP based on process analysis to simulate the actual production process and generate process data. The partial least squares method is then applied to develop predictive models of film thickness and economic efficiency. To manage the model uncertainty, the robust optimization approach is introduced to improve the feasibility of the optimized solution. Iterative learning control is then utilized to further refine the model uncertainty. The constrained film thickness is transformed into one of the tracked targets to overcome the drawback that traditional iterative learning control cannot address constraints. The goal setting of economic efficiency is updated continuously according to the film thickness setting until this reaches its desired value. Finally, fuzzy parameter adjustment is adopted to ensure that the economic efficiency and film thickness converge rapidly to their optimized values under the constraint conditions. The effectiveness of the proposed optimization control strategy is validated by simulation results.

The Use of Objective Structured Self-Assessment and Peer-Feedback (OSSP) for Learning Communication Skills: Evaluation Using a Controlled Trial

ERIC Educational Resources Information Center

Perera, Jennifer; Mohamadou, Galy; Kaur, Satpal

2010-01-01

Feedback is essential to guide students towards expected performance goals. The usefulness of teacher feedback on improving communication skills (CS) has been well documented. It has been proposed that self-assessment and peer-feedback has an equally important role to play in enhancing learning. This is the focus of this study. Objectively…

The Effects of Autonomy-Supportive and Controlling Teaching Behaviour in Biology Lessons with Primary and Secondary Experiences on Students' Intrinsic Motivation and Flow-Experience

ERIC Educational Resources Information Center

Hofferber, Natalia; Basten, Melanie; Großmann, Nadine; Wilde, Matthias

2016-01-01

Self-Determination Theory and Flow Theory propose that perceived autonomy fosters the positive qualities of motivation and flow-experience. Autonomy-support can help to maintain students' motivation in very interesting learning activities and may lead to an increase in the positive qualities of motivation in less interesting learning activities.…

Enhancing Self-Regulation and Approaches to Learning in First-Year College Students: A Narrative-Based Programme Assessed in the Iberian Peninsula

ERIC Educational Resources Information Center

Rosario, Pedro; Nunez, Jose C.; Gonzalez-Pienda, Julio; Valle, Antonio; Trigo, Luisa; Guimaraes, Carina

2010-01-01

A programme to enhance first-year college students' self-regulated learning strategies is proposed and assessed in two samples, one from the University of Oviedo (Spain) and the other from the University of Minho (Portugal). Each sample was divided into two groups (experimental and control). The Spanish sample comprised 44 students in the…

Modeling choice and reaction time during arbitrary visuomotor learning through the coordination of adaptive working memory and reinforcement learning

PubMed Central

Viejo, Guillaume; Khamassi, Mehdi; Brovelli, Andrea; Girard, Benoît

2015-01-01

Current learning theory provides a comprehensive description of how humans and other animals learn, and places behavioral flexibility and automaticity at heart of adaptive behaviors. However, the computations supporting the interactions between goal-directed and habitual decision-making systems are still poorly understood. Previous functional magnetic resonance imaging (fMRI) results suggest that the brain hosts complementary computations that may differentially support goal-directed and habitual processes in the form of a dynamical interplay rather than a serial recruitment of strategies. To better elucidate the computations underlying flexible behavior, we develop a dual-system computational model that can predict both performance (i.e., participants' choices) and modulations in reaction times during learning of a stimulus–response association task. The habitual system is modeled with a simple Q-Learning algorithm (QL). For the goal-directed system, we propose a new Bayesian Working Memory (BWM) model that searches for information in the history of previous trials in order to minimize Shannon entropy. We propose a model for QL and BWM coordination such that the expensive memory manipulation is under control of, among others, the level of convergence of the habitual learning. We test the ability of QL or BWM alone to explain human behavior, and compare them with the performance of model combinations, to highlight the need for such combinations to explain behavior. Two of the tested combination models are derived from the literature, and the latter being our new proposal. In conclusion, all subjects were better explained by model combinations, and the majority of them are explained by our new coordination proposal. PMID:26379518

Cooperation-Controlled Learning for Explicit Class Structure in Self-Organizing Maps

PubMed Central

Kamimura, Ryotaro

2014-01-01

We attempt to demonstrate the effectiveness of multiple points of view toward neural networks. By restricting ourselves to two points of view of a neuron, we propose a new type of information-theoretic method called “cooperation-controlled learning.” In this method, individual and collective neurons are distinguished from one another, and we suppose that the characteristics of individual and collective neurons are different. To implement individual and collective neurons, we prepare two networks, namely, cooperative and uncooperative networks. The roles of these networks and the roles of individual and collective neurons are controlled by the cooperation parameter. As the parameter is increased, the role of cooperative networks becomes more important in learning, and the characteristics of collective neurons become more dominant. On the other hand, when the parameter is small, individual neurons play a more important role. We applied the method to the automobile and housing data from the machine learning database and examined whether explicit class boundaries could be obtained. Experimental results showed that cooperation-controlled learning, in particular taking into account information on input units, could be used to produce clearer class structure than conventional self-organizing maps. PMID:25309950

Cognitive Fatigue Facilitates Procedural Sequence Learning.

PubMed

Borragán, Guillermo; Slama, Hichem; Destrebecqz, Arnaud; Peigneux, Philippe

2016-01-01

Enhanced procedural learning has been evidenced in conditions where cognitive control is diminished, including hypnosis, disruption of prefrontal activity and non-optimal time of the day. Another condition depleting the availability of controlled resources is cognitive fatigue (CF). We tested the hypothesis that CF, eventually leading to diminished cognitive control, facilitates procedural sequence learning. In a two-day experiment, 23 young healthy adults were administered a serial reaction time task (SRTT) following the induction of high or low levels of CF, in a counterbalanced order. CF was induced using the Time load Dual-back (TloadDback) paradigm, a dual working memory task that allows tailoring cognitive load levels to the individual's optimal performance capacity. In line with our hypothesis, reaction times (RT) in the SRTT were faster in the high- than in the low-level fatigue condition, and performance improvement was higher for the sequential than the motor components. Altogether, our results suggest a paradoxical, facilitating impact of CF on procedural motor sequence learning. We propose that facilitated learning in the high-level fatigue condition stems from a reduction in the cognitive resources devoted to cognitive control processes that normally oppose automatic procedural acquisition mechanisms.

Microgrid energy dispatching for industrial zones with renewable generations and electric vehicles via stochastic optimization and learning

NASA Astrophysics Data System (ADS)

Zhang, Kai; Li, Jingzhi; He, Zhubin; Yan, Wanfeng

2018-07-01

In this paper, a stochastic optimization framework is proposed to address the microgrid energy dispatching problem with random renewable generation and vehicle activity pattern, which is closer to the practical applications. The patterns of energy generation, consumption and storage availability are all random and unknown at the beginning, and the microgrid controller design (MCD) is formulated as a Markov decision process (MDP). Hence, an online learning-based control algorithm is proposed for the microgrid, which could adapt the control policy with increasing knowledge of the system dynamics and converges to the optimal algorithm. We adopt the linear approximation idea to decompose the original value functions as the summation of each per-battery value function. As a consequence, the computational complexity is significantly reduced from exponential growth to linear growth with respect to the size of battery states. Monte Carlo simulation of different scenarios demonstrates the effectiveness and efficiency of our algorithm.

Reinforcement learning of periodical gaits in locomotion robots

NASA Astrophysics Data System (ADS)

Svinin, Mikhail; Yamada, Kazuyaki; Ushio, S.; Ueda, Kanji

1999-08-01

Emergence of stable gaits in locomotion robots is studied in this paper. A classifier system, implementing an instance- based reinforcement learning scheme, is used for sensory- motor control of an eight-legged mobile robot. Important feature of the classifier system is its ability to work with the continuous sensor space. The robot does not have a prior knowledge of the environment, its own internal model, and the goal coordinates. It is only assumed that the robot can acquire stable gaits by learning how to reach a light source. During the learning process the control system, is self-organized by reinforcement signals. Reaching the light source defines a global reward. Forward motion gets a local reward, while stepping back and falling down get a local punishment. Feasibility of the proposed self-organized system is tested under simulation and experiment. The control actions are specified at the leg level. It is shown that, as learning progresses, the number of the action rules in the classifier systems is stabilized to a certain level, corresponding to the acquired gait patterns.

Career Success: The Effects of Personality.

ERIC Educational Resources Information Center

Lau, Victor P.; Shaffer, Margaret A.

1999-01-01

A model based on Bandura's Social Learning Theory proposes the following personality traits as determinants of career success: locus of control, self-monitoring, self-esteem, and optimism, along with job performance and person-to-environment fit. (SK)

An alternative approach based on artificial neural networks to study controlled drug release.

PubMed

Reis, Marcus A A; Sinisterra, Rubén D; Belchior, Jadson C

2004-02-01

An alternative methodology based on artificial neural networks is proposed to be a complementary tool to other conventional methods to study controlled drug release. Two systems are used to test the approach; namely, hydrocortisone in a biodegradable matrix and rhodium (II) butyrate complexes in a bioceramic matrix. Two well-established mathematical models are used to simulate different release profiles as a function of fundamental properties; namely, diffusion coefficient (D), saturation solubility (C(s)), drug loading (A), and the height of the device (h). The models were tested, and the results show that these fundamental properties can be predicted after learning the experimental or model data for controlled drug release systems. The neural network results obtained after the learning stage can be considered to quantitatively predict ideal experimental conditions. Overall, the proposed methodology was shown to be efficient for ideal experiments, with a relative average error of <1% in both tests. This approach can be useful for the experimental analysis to simulate and design efficient controlled drug-release systems. Copyright 2004 Wiley-Liss, Inc. and the American Pharmacists Association

Evolving a Behavioral Repertoire for a Walking Robot.

PubMed

Cully, A; Mouret, J-B

2016-01-01

Numerous algorithms have been proposed to allow legged robots to learn to walk. However, most of these algorithms are devised to learn walking in a straight line, which is not sufficient to accomplish any real-world mission. Here we introduce the Transferability-based Behavioral Repertoire Evolution algorithm (TBR-Evolution), a novel evolutionary algorithm that simultaneously discovers several hundreds of simple walking controllers, one for each possible direction. By taking advantage of solutions that are usually discarded by evolutionary processes, TBR-Evolution is substantially faster than independently evolving each controller. Our technique relies on two methods: (1) novelty search with local competition, which searches for both high-performing and diverse solutions, and (2) the transferability approach, which combines simulations and real tests to evolve controllers for a physical robot. We evaluate this new technique on a hexapod robot. Results show that with only a few dozen short experiments performed on the robot, the algorithm learns a repertoire of controllers that allows the robot to reach every point in its reachable space. Overall, TBR-Evolution introduced a new kind of learning algorithm that simultaneously optimizes all the achievable behaviors of a robot.

Swarm-wavelet based extreme learning machine for finger movement classification on transradial amputees.

PubMed

Anam, Khairul; Al-Jumaily, Adel

2014-01-01

The use of a small number of surface electromyography (EMG) channels on the transradial amputee in a myoelectric controller is a big challenge. This paper proposes a pattern recognition system using an extreme learning machine (ELM) optimized by particle swarm optimization (PSO). PSO is mutated by wavelet function to avoid trapped in a local minima. The proposed system is used to classify eleven imagined finger motions on five amputees by using only two EMG channels. The optimal performance of wavelet-PSO was compared to a grid-search method and standard PSO. The experimental results show that the proposed system is the most accurate classifier among other tested classifiers. It could classify 11 finger motions with the average accuracy of about 94 % across five amputees.

Reinforcement learning design-based adaptive tracking control with less learning parameters for nonlinear discrete-time MIMO systems.

PubMed

Liu, Yan-Jun; Tang, Li; Tong, Shaocheng; Chen, C L Philip; Li, Dong-Juan

2015-01-01

Based on the neural network (NN) approximator, an online reinforcement learning algorithm is proposed for a class of affine multiple input and multiple output (MIMO) nonlinear discrete-time systems with unknown functions and disturbances. In the design procedure, two networks are provided where one is an action network to generate an optimal control signal and the other is a critic network to approximate the cost function. An optimal control signal and adaptation laws can be generated based on two NNs. In the previous approaches, the weights of critic and action networks are updated based on the gradient descent rule and the estimations of optimal weight vectors are directly adjusted in the design. Consequently, compared with the existing results, the main contributions of this paper are: 1) only two parameters are needed to be adjusted, and thus the number of the adaptation laws is smaller than the previous results and 2) the updating parameters do not depend on the number of the subsystems for MIMO systems and the tuning rules are replaced by adjusting the norms on optimal weight vectors in both action and critic networks. It is proven that the tracking errors, the adaptation laws, and the control inputs are uniformly bounded using Lyapunov analysis method. The simulation examples are employed to illustrate the effectiveness of the proposed algorithm.

A learning-based semi-autonomous controller for robotic exploration of unknown disaster scenes while searching for victims.

PubMed

Doroodgar, Barzin; Liu, Yugang; Nejat, Goldie

2014-12-01

Semi-autonomous control schemes can address the limitations of both teleoperation and fully autonomous robotic control of rescue robots in disaster environments by allowing a human operator to cooperate and share such tasks with a rescue robot as navigation, exploration, and victim identification. In this paper, we present a unique hierarchical reinforcement learning-based semi-autonomous control architecture for rescue robots operating in cluttered and unknown urban search and rescue (USAR) environments. The aim of the controller is to enable a rescue robot to continuously learn from its own experiences in an environment in order to improve its overall performance in exploration of unknown disaster scenes. A direction-based exploration technique is integrated in the controller to expand the search area of the robot via the classification of regions and the rubble piles within these regions. Both simulations and physical experiments in USAR-like environments verify the robustness of the proposed HRL-based semi-autonomous controller to unknown cluttered scenes with different sizes and varying types of configurations.

Biomimetic Hybrid Feedback Feedforward Neural-Network Learning Control.

PubMed

Pan, Yongping; Yu, Haoyong

2017-06-01

This brief presents a biomimetic hybrid feedback feedforward neural-network learning control (NNLC) strategy inspired by the human motor learning control mechanism for a class of uncertain nonlinear systems. The control structure includes a proportional-derivative controller acting as a feedback servo machine and a radial-basis-function (RBF) NN acting as a feedforward predictive machine. Under the sufficient constraints on control parameters, the closed-loop system achieves semiglobal practical exponential stability, such that an accurate NN approximation is guaranteed in a local region along recurrent reference trajectories. Compared with the existing NNLC methods, the novelties of the proposed method include: 1) the implementation of an adaptive NN control to guarantee plant states being recurrent is not needed, since recurrent reference signals rather than plant states are utilized as NN inputs, which greatly simplifies the analysis and synthesis of the NNLC and 2) the domain of NN approximation can be determined a priori by the given reference signals, which leads to an easy construction of the RBF-NNs. Simulation results have verified the effectiveness of this approach.

Hybrid supervisory control using recurrent fuzzy neural network for tracking periodic inputs.

PubMed

Lin, F J; Wai, R J; Hong, C M

2001-01-01

A hybrid supervisory control system using a recurrent fuzzy neural network (RFNN) is proposed to control the mover of a permanent magnet linear synchronous motor (PMLSM) servo drive for the tracking of periodic reference inputs. First, the field-oriented mechanism is applied to formulate the dynamic equation of the PMLSM. Then, a hybrid supervisory control system, which combines a supervisory control system and an intelligent control system, is proposed to control the mover of the PMLSM for periodic motion. The supervisory control law is designed based on the uncertainty bounds of the controlled system to stabilize the system states around a predefined bound region. Since the supervisory control law will induce excessive and chattering control effort, the intelligent control system is introduced to smooth and reduce the control effort when the system states are inside the predefined bound region. In the intelligent control system, the RFNN control is the main tracking controller which is used to mimic a idea control law and a compensated control is proposed to compensate the difference between the idea control law and the RFNN control. The RFNN has the merits of fuzzy inference, dynamic mapping and fast convergence speed, In addition, an online parameter training methodology, which is derived using the Lyapunov stability theorem and the gradient descent method, is proposed to increase the learning capability of the RFNN. The proposed hybrid supervisory control system using RFNN can track various periodic reference inputs effectively with robust control performance.

Action-based language: a theory of language acquisition, comprehension, and production.

PubMed

Glenberg, Arthur M; Gallese, Vittorio

2012-07-01

Evolution and the brain have done a marvelous job solving many tricky problems in action control, including problems of learning, hierarchical control over serial behavior, continuous recalibration, and fluency in the face of slow feedback. Given that evolution tends to be conservative, it should not be surprising that these solutions are exploited to solve other tricky problems, such as the design of a communication system. We propose that a mechanism of motor control, paired controller/predictor models, has been exploited for language learning, comprehension, and production. Our account addresses the development of grammatical regularities and perspective, as well as how linguistic symbols become meaningful through grounding in perception, action, and emotional systems. Copyright © 2011 Elsevier Srl. All rights reserved.

Fuzzy CMAC With incremental Bayesian Ying-Yang learning and dynamic rule construction.

PubMed

Nguyen, M N

2010-04-01

Inspired by the philosophy of ancient Chinese Taoism, Xu's Bayesian ying-yang (BYY) learning technique performs clustering by harmonizing the training data (yang) with the solution (ying). In our previous work, the BYY learning technique was applied to a fuzzy cerebellar model articulation controller (FCMAC) to find the optimal fuzzy sets; however, this is not suitable for time series data analysis. To address this problem, we propose an incremental BYY learning technique in this paper, with the idea of sliding window and rule structure dynamic algorithms. Three contributions are made as a result of this research. First, an online expectation-maximization algorithm incorporated with the sliding window is proposed for the fuzzification phase. Second, the memory requirement is greatly reduced since the entire data set no longer needs to be obtained during the prediction process. Third, the rule structure dynamic algorithm with dynamically initializing, recruiting, and pruning rules relieves the "curse of dimensionality" problem that is inherent in the FCMAC. Because of these features, the experimental results of the benchmark data sets of currency exchange rates and Mackey-Glass show that the proposed model is more suitable for real-time streaming data analysis.

Off-Policy Actor-Critic Structure for Optimal Control of Unknown Systems With Disturbances.

PubMed

Song, Ruizhuo; Lewis, Frank L; Wei, Qinglai; Zhang, Huaguang

2016-05-01

An optimal control method is developed for unknown continuous-time systems with unknown disturbances in this paper. The integral reinforcement learning (IRL) algorithm is presented to obtain the iterative control. Off-policy learning is used to allow the dynamics to be completely unknown. Neural networks are used to construct critic and action networks. It is shown that if there are unknown disturbances, off-policy IRL may not converge or may be biased. For reducing the influence of unknown disturbances, a disturbances compensation controller is added. It is proven that the weight errors are uniformly ultimately bounded based on Lyapunov techniques. Convergence of the Hamiltonian function is also proven. The simulation study demonstrates the effectiveness of the proposed optimal control method for unknown systems with disturbances.

Functional Based Adaptive and Fuzzy Sliding Controller for Non-Autonomous Active Suspension System

NASA Astrophysics Data System (ADS)

Huang, Shiuh-Jer; Chen, Hung-Yi

In this paper, an adaptive sliding controller is developed for controlling a vehicle active suspension system. The functional approximation technique is employed to substitute the unknown non-autonomous functions of the suspension system and release the model-based requirement of sliding mode control algorithm. In order to improve the control performance and reduce the implementation problem, a fuzzy strategy with online learning ability is added to compensate the functional approximation error. The update laws of the functional approximation coefficients and the fuzzy tuning parameters are derived from the Lyapunov theorem to guarantee the system stability. The proposed controller is implemented on a quarter-car hydraulic actuating active suspension system test-rig. The experimental results show that the proposed controller suppresses the oscillation amplitude of the suspension system effectively.

A Message Exchange Protocol in Command and Control Systems Integration, using the JC3IEDM

DTIC Science & Technology

2014-06-01

19TH International Command and Control Research and Technology Symposium C2 Agility: Lessons Learned from Research and Operations. A Message...distribution unlimited 13. SUPPLEMENTARY NOTES Presented at the 18th International Command & Control Research & Technology Symposium (ICCRTS) held 16...presents approaches of integration, compares their technologies , points out their advantages, proposes requirements, and provides the design of a protocol

Intelligent voltage control strategy for three-phase UPS inverters with output LC filter

NASA Astrophysics Data System (ADS)

Jung, J. W.; Leu, V. Q.; Dang, D. Q.; Do, T. D.; Mwasilu, F.; Choi, H. H.

2015-08-01

This paper presents a supervisory fuzzy neural network control (SFNNC) method for a three-phase inverter of uninterruptible power supplies (UPSs). The proposed voltage controller is comprised of a fuzzy neural network control (FNNC) term and a supervisory control term. The FNNC term is deliberately employed to estimate the uncertain terms, and the supervisory control term is designed based on the sliding mode technique to stabilise the system dynamic errors. To improve the learning capability, the FNNC term incorporates an online parameter training methodology, using the gradient descent method and Lyapunov stability theory. Besides, a linear load current observer that estimates the load currents is used to exclude the load current sensors. The proposed SFNN controller and the observer are robust to the filter inductance variations, and their stability analyses are described in detail. The experimental results obtained on a prototype UPS test bed with a TMS320F28335 DSP are presented to validate the feasibility of the proposed scheme. Verification results demonstrate that the proposed control strategy can achieve smaller steady-state error and lower total harmonic distortion when subjected to nonlinear or unbalanced loads compared to the conventional sliding mode control method.

App-based serious gaming for training of chest tube insertion: study protocol for a randomized controlled trial.

PubMed

Friedrich, Mirco; Bergdolt, Christian; Haubruck, Patrick; Bruckner, Thomas; Kowalewski, Karl-Friedrich; Müller-Stich, Beat Peter; Tanner, Michael C; Nickel, Felix

2017-02-06

Chest tube insertion is a standard intervention for management of various injuries of the thorax. Quick and accurate execution facilitates efficient therapy without further complications. Here, we propose a new training concept comprised of e-learning elements as well as continuous rating using an objective structured assessment of technical skills (OSATS) tool. The study protocol is presented for a randomized trial to evaluate e-learning with app-based serious gaming for chest drain insertion. The proposed randomized trial will be carried out at the Department of Orthopedics and Traumatology at Heidelberg University in the context of regular curricular teaching for medical students (n = 90, 3rd to 6th year). The intervention group will use e-learning with the serious gaming app Touch Surgery (TM) for chest drain insertion, whereas the control group uses serious gaming for an unrelated procedure. Primary endpoint is operative performance of chest drain insertion in a porcine cadaveric model according to OSATS. The randomized trial will help determine the value of e-learning with the serious gaming app Touch Surgery (TM) for chest drain insertion by using the OSATS score. The study will improve surgical training for trauma situations. Trial Registration Number, DRKS00009994 . Registered on 27 May 2016.

SCADA-based Operator Support System for Power Plant Equipment Fault Forecasting

NASA Astrophysics Data System (ADS)

Mayadevi, N.; Ushakumari, S. S.; Vinodchandra, S. S.

2014-12-01

Power plant equipment must be monitored closely to prevent failures from disrupting plant availability. Online monitoring technology integrated with hybrid forecasting techniques can be used to prevent plant equipment faults. A self learning rule-based expert system is proposed in this paper for fault forecasting in power plants controlled by supervisory control and data acquisition (SCADA) system. Self-learning utilizes associative data mining algorithms on the SCADA history database to form new rules that can dynamically update the knowledge base of the rule-based expert system. In this study, a number of popular associative learning algorithms are considered for rule formation. Data mining results show that the Tertius algorithm is best suited for developing a learning engine for power plants. For real-time monitoring of the plant condition, graphical models are constructed by K-means clustering. To build a time-series forecasting model, a multi layer preceptron (MLP) is used. Once created, the models are updated in the model library to provide an adaptive environment for the proposed system. Graphical user interface (GUI) illustrates the variation of all sensor values affecting a particular alarm/fault, as well as the step-by-step procedure for avoiding critical situations and consequent plant shutdown. The forecasting performance is evaluated by computing the mean absolute error and root mean square error of the predictions.

Novel real-time tumor-contouring method using deep learning to prevent mistracking in X-ray fluoroscopy.

PubMed

Terunuma, Toshiyuki; Tokui, Aoi; Sakae, Takeji

2018-03-01

Robustness to obstacles is the most important factor necessary to achieve accurate tumor tracking without fiducial markers. Some high-density structures, such as bone, are enhanced on X-ray fluoroscopic images, which cause tumor mistracking. Tumor tracking should be performed by controlling "importance recognition": the understanding that soft-tissue is an important tracking feature and bone structure is unimportant. We propose a new real-time tumor-contouring method that uses deep learning with importance recognition control. The novelty of the proposed method is the combination of the devised random overlay method and supervised deep learning to induce the recognition of structures in tumor contouring as important or unimportant. This method can be used for tumor contouring because it uses deep learning to perform image segmentation. Our results from a simulated fluoroscopy model showed accurate tracking of a low-visibility tumor with an error of approximately 1 mm, even if enhanced bone structure acted as an obstacle. A high similarity of approximately 0.95 on the Jaccard index was observed between the segmented and ground truth tumor regions. A short processing time of 25 ms was achieved. The results of this simulated fluoroscopy model support the feasibility of robust real-time tumor contouring with fluoroscopy. Further studies using clinical fluoroscopy are highly anticipated.

Can model-free reinforcement learning explain deontological moral judgments?

PubMed

Ayars, Alisabeth

2016-05-01

Dual-systems frameworks propose that moral judgments are derived from both an immediate emotional response, and controlled/rational cognition. Recently Cushman (2013) proposed a new dual-system theory based on model-free and model-based reinforcement learning. Model-free learning attaches values to actions based on their history of reward and punishment, and explains some deontological, non-utilitarian judgments. Model-based learning involves the construction of a causal model of the world and allows for far-sighted planning; this form of learning fits well with utilitarian considerations that seek to maximize certain kinds of outcomes. I present three concerns regarding the use of model-free reinforcement learning to explain deontological moral judgment. First, many actions that humans find aversive from model-free learning are not judged to be morally wrong. Moral judgment must require something in addition to model-free learning. Second, there is a dearth of evidence for central predictions of the reinforcement account-e.g., that people with different reinforcement histories will, all else equal, make different moral judgments. Finally, to account for the effect of intention within the framework requires certain assumptions which lack support. These challenges are reasonable foci for future empirical/theoretical work on the model-free/model-based framework. Copyright © 2016 Elsevier B.V. All rights reserved.

An Adaptive Supervisory Sliding Fuzzy Cerebellar Model Articulation Controller for Sensorless Vector-Controlled Induction Motor Drive Systems

PubMed Central

Wang, Shun-Yuan; Tseng, Chwan-Lu; Lin, Shou-Chuang; Chiu, Chun-Jung; Chou, Jen-Hsiang

2015-01-01

This paper presents the implementation of an adaptive supervisory sliding fuzzy cerebellar model articulation controller (FCMAC) in the speed sensorless vector control of an induction motor (IM) drive system. The proposed adaptive supervisory sliding FCMAC comprised a supervisory controller, integral sliding surface, and an adaptive FCMAC. The integral sliding surface was employed to eliminate steady-state errors and enhance the responsiveness of the system. The adaptive FCMAC incorporated an FCMAC with a compensating controller to perform a desired control action. The proposed controller was derived using the Lyapunov approach, which guarantees learning-error convergence. The implementation of three intelligent control schemes—the adaptive supervisory sliding FCMAC, adaptive sliding FCMAC, and adaptive sliding CMAC—were experimentally investigated under various conditions in a realistic sensorless vector-controlled IM drive system. The root mean square error (RMSE) was used as a performance index to evaluate the experimental results of each control scheme. The analysis results indicated that the proposed adaptive supervisory sliding FCMAC substantially improved the system performance compared with the other control schemes. PMID:25815450

An adaptive supervisory sliding fuzzy cerebellar model articulation controller for sensorless vector-controlled induction motor drive systems.

PubMed

Wang, Shun-Yuan; Tseng, Chwan-Lu; Lin, Shou-Chuang; Chiu, Chun-Jung; Chou, Jen-Hsiang

2015-03-25

This paper presents the implementation of an adaptive supervisory sliding fuzzy cerebellar model articulation controller (FCMAC) in the speed sensorless vector control of an induction motor (IM) drive system. The proposed adaptive supervisory sliding FCMAC comprised a supervisory controller, integral sliding surface, and an adaptive FCMAC. The integral sliding surface was employed to eliminate steady-state errors and enhance the responsiveness of the system. The adaptive FCMAC incorporated an FCMAC with a compensating controller to perform a desired control action. The proposed controller was derived using the Lyapunov approach, which guarantees learning-error convergence. The implementation of three intelligent control schemes--the adaptive supervisory sliding FCMAC, adaptive sliding FCMAC, and adaptive sliding CMAC--were experimentally investigated under various conditions in a realistic sensorless vector-controlled IM drive system. The root mean square error (RMSE) was used as a performance index to evaluate the experimental results of each control scheme. The analysis results indicated that the proposed adaptive supervisory sliding FCMAC substantially improved the system performance compared with the other control schemes.

Decentralized learning in Markov games.

PubMed

Vrancx, Peter; Verbeeck, Katja; Nowé, Ann

2008-08-01

Learning automata (LA) were recently shown to be valuable tools for designing multiagent reinforcement learning algorithms. One of the principal contributions of the LA theory is that a set of decentralized independent LA is able to control a finite Markov chain with unknown transition probabilities and rewards. In this paper, we propose to extend this algorithm to Markov games--a straightforward extension of single-agent Markov decision problems to distributed multiagent decision problems. We show that under the same ergodic assumptions of the original theorem, the extended algorithm will converge to a pure equilibrium point between agent policies.

Reinforcement learning controller design for affine nonlinear discrete-time systems using online approximators.

PubMed

Yang, Qinmin; Jagannathan, Sarangapani

2012-04-01

In this paper, reinforcement learning state- and output-feedback-based adaptive critic controller designs are proposed by using the online approximators (OLAs) for a general multi-input and multioutput affine unknown nonlinear discretetime systems in the presence of bounded disturbances. The proposed controller design has two entities, an action network that is designed to produce optimal signal and a critic network that evaluates the performance of the action network. The critic estimates the cost-to-go function which is tuned online using recursive equations derived from heuristic dynamic programming. Here, neural networks (NNs) are used both for the action and critic whereas any OLAs, such as radial basis functions, splines, fuzzy logic, etc., can be utilized. For the output-feedback counterpart, an additional NN is designated as the observer to estimate the unavailable system states, and thus, separation principle is not required. The NN weight tuning laws for the controller schemes are also derived while ensuring uniform ultimate boundedness of the closed-loop system using Lyapunov theory. Finally, the effectiveness of the two controllers is tested in simulation on a pendulum balancing system and a two-link robotic arm system.

Central Nervous System Control of Voice and Swallowing

PubMed Central

Ludlow, Christy L.

2015-01-01

This review of the central nervous control systems for voice and swallowing has suggested that the traditional concepts of a separation between cortical and limbic and brain stem control should be refined and more integrative. For voice production, a separation of the non-human vocalization system from the human learned voice production system has been posited based primarily on studies of non-human primates. However, recent humans studies of emotionally based vocalizations and human volitional voice production has shown more integration between these two systems than previously proposed. Recent human studies have shown that reflexive vocalization as well as learned voice production not involving speech, involve a common integrative system. On the other hand, recent studies of non-human primates have provided evidence of some cortical activity during vocalization and cortical changes with training during vocal behavior. For swallowing, evidence from the macaque and functional brain imaging in humans indicates that the control for the pharyngeal phase of swallowing is not primarily under brain stem mechanisms as previously proposed. Studies suggest that the initiation and patterning of swallowing for the pharyngeal phase is also under active cortical control for both spontaneous as well as volitional swallowing in awake humans and non-human primates. PMID:26241238

Actor-critic-based optimal tracking for partially unknown nonlinear discrete-time systems.

PubMed

Kiumarsi, Bahare; Lewis, Frank L

2015-01-01

This paper presents a partially model-free adaptive optimal control solution to the deterministic nonlinear discrete-time (DT) tracking control problem in the presence of input constraints. The tracking error dynamics and reference trajectory dynamics are first combined to form an augmented system. Then, a new discounted performance function based on the augmented system is presented for the optimal nonlinear tracking problem. In contrast to the standard solution, which finds the feedforward and feedback terms of the control input separately, the minimization of the proposed discounted performance function gives both feedback and feedforward parts of the control input simultaneously. This enables us to encode the input constraints into the optimization problem using a nonquadratic performance function. The DT tracking Bellman equation and tracking Hamilton-Jacobi-Bellman (HJB) are derived. An actor-critic-based reinforcement learning algorithm is used to learn the solution to the tracking HJB equation online without requiring knowledge of the system drift dynamics. That is, two neural networks (NNs), namely, actor NN and critic NN, are tuned online and simultaneously to generate the optimal bounded control policy. A simulation example is given to show the effectiveness of the proposed method.

Reinforcement Learning and Dopamine in Schizophrenia: Dimensions of Symptoms or Specific Features of a Disease Group?

PubMed Central

Deserno, Lorenz; Boehme, Rebecca; Heinz, Andreas; Schlagenhauf, Florian

2013-01-01

Abnormalities in reinforcement learning are a key finding in schizophrenia and have been proposed to be linked to elevated levels of dopamine neurotransmission. Behavioral deficits in reinforcement learning and their neural correlates may contribute to the formation of clinical characteristics of schizophrenia. The ability to form predictions about future outcomes is fundamental for environmental interactions and depends on neuronal teaching signals, like reward prediction errors. While aberrant prediction errors, that encode non-salient events as surprising, have been proposed to contribute to the formation of positive symptoms, a failure to build neural representations of decision values may result in negative symptoms. Here, we review behavioral and neuroimaging research in schizophrenia and focus on studies that implemented reinforcement learning models. In addition, we discuss studies that combined reinforcement learning with measures of dopamine. Thereby, we suggest how reinforcement learning abnormalities in schizophrenia may contribute to the formation of psychotic symptoms and may interact with cognitive deficits. These ideas point toward an interplay of more rigid versus flexible control over reinforcement learning. Pronounced deficits in the flexible or model-based domain may allow for a detailed characterization of well-established cognitive deficits in schizophrenia patients based on computational models of learning. Finally, we propose a framework based on the potentially crucial contribution of dopamine to dysfunctional reinforcement learning on the level of neural networks. Future research may strongly benefit from computational modeling but also requires further methodological improvement for clinical group studies. These research tools may help to improve our understanding of disease-specific mechanisms and may help to identify clinically relevant subgroups of the heterogeneous entity schizophrenia. PMID:24391603

REVIEW: Internal models in sensorimotor integration: perspectives from adaptive control theory

NASA Astrophysics Data System (ADS)

Tin, Chung; Poon, Chi-Sang

2005-09-01

Internal models and adaptive controls are empirical and mathematical paradigms that have evolved separately to describe learning control processes in brain systems and engineering systems, respectively. This paper presents a comprehensive appraisal of the correlation between these paradigms with a view to forging a unified theoretical framework that may benefit both disciplines. It is suggested that the classic equilibrium-point theory of impedance control of arm movement is analogous to continuous gain-scheduling or high-gain adaptive control within or across movement trials, respectively, and that the recently proposed inverse internal model is akin to adaptive sliding control originally for robotic manipulator applications. Modular internal models' architecture for multiple motor tasks is a form of multi-model adaptive control. Stochastic methods, such as generalized predictive control, reinforcement learning, Bayesian learning and Hebbian feedback covariance learning, are reviewed and their possible relevance to motor control is discussed. Possible applicability of a Luenberger observer and an extended Kalman filter to state estimation problems—such as sensorimotor prediction or the resolution of vestibular sensory ambiguity—is also discussed. The important role played by vestibular system identification in postural control suggests an indirect adaptive control scheme whereby system states or parameters are explicitly estimated prior to the implementation of control. This interdisciplinary framework should facilitate the experimental elucidation of the mechanisms of internal models in sensorimotor systems and the reverse engineering of such neural mechanisms into novel brain-inspired adaptive control paradigms in future.

Hierarchical control of procedural and declarative category-learning systems

PubMed Central

Turner, Benjamin O.; Crossley, Matthew J.; Ashby, F. Gregory

2017-01-01

Substantial evidence suggests that human category learning is governed by the interaction of multiple qualitatively distinct neural systems. In this view, procedural memory is used to learn stimulus-response associations, and declarative memory is used to apply explicit rules and test hypotheses about category membership. However, much less is known about the interaction between these systems: how is control passed between systems as they interact to influence motor resources? Here, we used fMRI to elucidate the neural correlates of switching between procedural and declarative categorization systems. We identified a key region of the cerebellum (left Crus I) whose activity was bidirectionally modulated depending on switch direction. We also identified regions of the default mode network (DMN) that were selectively connected to left Crus I during switching. We propose that the cerebellum—in coordination with the DMN—serves a critical role in passing control between procedural and declarative memory systems. PMID:28213114

Composite Intelligent Learning Control of Strict-Feedback Systems With Disturbance.

PubMed

Xu, Bin; Sun, Fuchun

2018-02-01

This paper addresses the dynamic surface control of uncertain nonlinear systems on the basis of composite intelligent learning and disturbance observer in presence of unknown system nonlinearity and time-varying disturbance. The serial-parallel estimation model with intelligent approximation and disturbance estimation is built to obtain the prediction error and in this way the composite law for weights updating is constructed. The nonlinear disturbance observer is developed using intelligent approximation information while the disturbance estimation is guaranteed to converge to a bounded compact set. The highlight is that different from previous work directly toward asymptotic stability, the transparency of the intelligent approximation and disturbance estimation is included in the control scheme. The uniformly ultimate boundedness stability is analyzed via Lyapunov method. Through simulation verification, the composite intelligent learning with disturbance observer can efficiently estimate the effect caused by system nonlinearity and disturbance while the proposed approach obtains better performance with higher accuracy.

Reinforcement learning solution for HJB equation arising in constrained optimal control problem.

PubMed

Luo, Biao; Wu, Huai-Ning; Huang, Tingwen; Liu, Derong

2015-11-01

The constrained optimal control problem depends on the solution of the complicated Hamilton-Jacobi-Bellman equation (HJBE). In this paper, a data-based off-policy reinforcement learning (RL) method is proposed, which learns the solution of the HJBE and the optimal control policy from real system data. One important feature of the off-policy RL is that its policy evaluation can be realized with data generated by other behavior policies, not necessarily the target policy, which solves the insufficient exploration problem. The convergence of the off-policy RL is proved by demonstrating its equivalence to the successive approximation approach. Its implementation procedure is based on the actor-critic neural networks structure, where the function approximation is conducted with linearly independent basis functions. Subsequently, the convergence of the implementation procedure with function approximation is also proved. Finally, its effectiveness is verified through computer simulations. Copyright © 2015 Elsevier Ltd. All rights reserved.

Fuzzy Sarsa with Focussed Replacing Eligibility Traces for Robust and Accurate Control

NASA Astrophysics Data System (ADS)

Kamdem, Sylvain; Ohki, Hidehiro; Sueda, Naomichi

Several methods of reinforcement learning in continuous state and action spaces that utilize fuzzy logic have been proposed in recent years. This paper introduces Fuzzy Sarsa(λ), an on-policy algorithm for fuzzy learning that relies on a novel way of computing replacing eligibility traces to accelerate the policy evaluation. It is tested against several temporal difference learning algorithms: Sarsa(λ), Fuzzy Q(λ), an earlier fuzzy version of Sarsa and an actor-critic algorithm. We perform detailed evaluations on two benchmark problems : a maze domain and the cart pole. Results of various tests highlight the strengths and weaknesses of these algorithms and show that Fuzzy Sarsa(λ) outperforms all other algorithms tested for a larger granularity of design and under noisy conditions. It is a highly competitive method of learning in realistic noisy domains where a denser fuzzy design over the state space is needed for a more precise control.

REWARD/PUNISHMENT REVERSAL LEARNING IN OLDER SUICIDE ATTEMPTERS

PubMed Central

Dombrovski, Alexandre Y.; Clark, Luke; Siegle, Greg J.; Butters, Meryl A.; Ichikawa, Naho; Sahakian, Barbara; Szanto, Katalin

2011-01-01

Objective Suicide rates are very high in old age, and the contribution of cognitive risk factors remains poorly understood. Suicide may be viewed as an outcome of an altered decision process. We hypothesized that impairment in a component of affective decision-making – reward/punishment-based learning – is associated with attempted suicide in late-life depression. We expected that suicide attempters would discount past reward/punishment history, focusing excessively on the most recent rewards and punishments. Further, we hypothesized that this impairment could be dissociated from executive abilities such as forward planning. Method We assessed reward/punishment-based learning using the Probabilistic Reversal Learning task in 65 individuals aged 60 and older: suicide attempters, suicide ideators, non-suicidal depressed elderly, and non-depressed controls. We used a reinforcement learning computational model to decompose reward/punishment processing over time. The Stockings of Cambridge test served as a control measure of executive function. Results Suicide attempters but not suicide ideators showed impaired probabilistic reversal learning compared to both non-suicidal depressed elderly and to non-depressed controls, after controlling for effects of education, global cognitive function, and substance use. Model-based analyses revealed that suicide attempters discounted previous history to a higher degree, compared to controls, basing their choice largely on reward/punishment received on the last trial. Groups did not differ in their performance on the Stockings of Cambridge. Conclusions Older suicide attempters display impaired reward/punishment-based learning. We propose a hypothesis that older suicide attempters make overly present-focused decisions, ignoring past experiences. Modification of this ‘myopia for the past’ may have therapeutic potential. PMID:20231320

Divide et impera: subgoaling reduces the complexity of probabilistic inference and problem solving

PubMed Central

Maisto, Domenico; Donnarumma, Francesco; Pezzulo, Giovanni

2015-01-01

It has long been recognized that humans (and possibly other animals) usually break problems down into smaller and more manageable problems using subgoals. Despite a general consensus that subgoaling helps problem solving, it is still unclear what the mechanisms guiding online subgoal selection are during the solution of novel problems for which predefined solutions are not available. Under which conditions does subgoaling lead to optimal behaviour? When is subgoaling better than solving a problem from start to finish? Which is the best number and sequence of subgoals to solve a given problem? How are these subgoals selected during online inference? Here, we present a computational account of subgoaling in problem solving. Following Occam's razor, we propose that good subgoals are those that permit planning solutions and controlling behaviour using less information resources, thus yielding parsimony in inference and control. We implement this principle using approximate probabilistic inference: subgoals are selected using a sampling method that considers the descriptive complexity of the resulting sub-problems. We validate the proposed method using a standard reinforcement learning benchmark (four-rooms scenario) and show that the proposed method requires less inferential steps and permits selecting more compact control programs compared to an equivalent procedure without subgoaling. Furthermore, we show that the proposed method offers a mechanistic explanation of the neuronal dynamics found in the prefrontal cortex of monkeys that solve planning problems. Our computational framework provides a novel integrative perspective on subgoaling and its adaptive advantages for planning, control and learning, such as for example lowering cognitive effort and working memory load. PMID:25652466

Improvements from a Flipped Classroom May Simply Be the Fruits of Active Learning

PubMed Central

Jensen, Jamie L.; Kummer, Tyler A.; Godoy, Patricia D. d. M.

2015-01-01

The “flipped classroom” is a learning model in which content attainment is shifted forward to outside of class, then followed by instructor-facilitated concept application activities in class. Current studies on the flipped model are limited. Our goal was to provide quantitative and controlled data about the effectiveness of this model. Using a quasi-experimental design, we compared an active nonflipped classroom with an active flipped classroom, both using the 5-E learning cycle, in an effort to vary only the role of the instructor and control for as many of the other potentially influential variables as possible. Results showed that both low-level and deep conceptual learning were equivalent between the conditions. Attitudinal data revealed equal student satisfaction with the course. Interestingly, both treatments ranked their contact time with the instructor as more influential to their learning than what they did at home. We conclude that the flipped classroom does not result in higher learning gains or better attitudes compared with the nonflipped classroom when both utilize an active-learning, constructivist approach and propose that learning gains in either condition are most likely a result of the active-learning style of instruction rather than the order in which the instructor participated in the learning process. PMID:25699543

A delivery mode study: The effect of self-paced video learning on first-year college students' achievement in calculus

NASA Astrophysics Data System (ADS)

Oktaviyanthi, Rina; Herman, Tatang

2016-10-01

In this paper, the effect of two different modes of deliver are proposed. The use of self-paced video learning and conventional learning methods in mathematics are compared. The research design classified as a quasi-experiment. The participants were 80 students in the first-year college and divided into two groups. One group as an experiment class received self-paced video learning method and the other group as a control group taught by conventional learning method. Pre and posttest were employed to measure the students' achievement, while questionnaire and interviews were applied to support the pre and posttest data. Statistical analysis included the independent samples t-test showed differences (p < 0.05) in posttest between the experimental and control groups, it means that the use of self-paced video contributed on students' achievement and students' attitudes. In addition, related to corresponding to the students' answer, there are five positive gains in using self-paced video in learning Calculus, such as appropriate learning for both audio and visual of students' characteristics, useful to learn Calculus, assisting students to be more engaging and paying attention in learning, helping students in making the concepts of Calculus are visible, interesting media and motivating students to learn independently.

An Efficient Interval Type-2 Fuzzy CMAC for Chaos Time-Series Prediction and Synchronization.

PubMed

Lee, Ching-Hung; Chang, Feng-Yu; Lin, Chih-Min

2014-03-01

This paper aims to propose a more efficient control algorithm for chaos time-series prediction and synchronization. A novel type-2 fuzzy cerebellar model articulation controller (T2FCMAC) is proposed. In some special cases, this T2FCMAC can be reduced to an interval type-2 fuzzy neural network, a fuzzy neural network, and a fuzzy cerebellar model articulation controller (CMAC). So, this T2FCMAC is a more generalized network with better learning ability, thus, it is used for the chaos time-series prediction and synchronization. Moreover, this T2FCMAC realizes the un-normalized interval type-2 fuzzy logic system based on the structure of the CMAC. It can provide better capabilities for handling uncertainty and more design degree of freedom than traditional type-1 fuzzy CMAC. Unlike most of the interval type-2 fuzzy system, the type-reduction of T2FCMAC is bypassed due to the property of un-normalized interval type-2 fuzzy logic system. This causes T2FCMAC to have lower computational complexity and is more practical. For chaos time-series prediction and synchronization applications, the training architectures with corresponding convergence analyses and optimal learning rates based on Lyapunov stability approach are introduced. Finally, two illustrated examples are presented to demonstrate the performance of the proposed T2FCMAC.

Automatic labeling of MR brain images through extensible learning and atlas forests.

PubMed

Xu, Lijun; Liu, Hong; Song, Enmin; Yan, Meng; Jin, Renchao; Hung, Chih-Cheng

2017-12-01

Multiatlas-based method is extensively used in MR brain images segmentation because of its simplicity and robustness. This method provides excellent accuracy although it is time consuming and limited in terms of obtaining information about new atlases. In this study, an automatic labeling of MR brain images through extensible learning and atlas forest is presented to address these limitations. We propose an extensible learning model which allows the multiatlas-based framework capable of managing the datasets with numerous atlases or dynamic atlas datasets and simultaneously ensure the accuracy of automatic labeling. Two new strategies are used to reduce the time and space complexity and improve the efficiency of the automatic labeling of brain MR images. First, atlases are encoded to atlas forests through random forest technology to reduce the time consumed for cross-registration between atlases and target image, and a scatter spatial vector is designed to eliminate errors caused by inaccurate registration. Second, an atlas selection method based on the extensible learning model is used to select atlases for target image without traversing the entire dataset and then obtain the accurate labeling. The labeling results of the proposed method were evaluated in three public datasets, namely, IBSR, LONI LPBA40, and ADNI. With the proposed method, the dice coefficient metric values on the three datasets were 84.17 ± 4.61%, 83.25 ± 4.29%, and 81.88 ± 4.53% which were 5% higher than those of the conventional method, respectively. The efficiency of the extensible learning model was evaluated by state-of-the-art methods for labeling of MR brain images. Experimental results showed that the proposed method could achieve accurate labeling for MR brain images without traversing the entire datasets. In the proposed multiatlas-based method, extensible learning and atlas forests were applied to control the automatic labeling of brain anatomies on large atlas datasets or dynamic atlas datasets and obtain accurate results. © 2017 American Association of Physicists in Medicine.

Representing high-dimensional data to intelligent prostheses and other wearable assistive robots: A first comparison of tile coding and selective Kanerva coding.

PubMed

Travnik, Jaden B; Pilarski, Patrick M

2017-07-01

Prosthetic devices have advanced in their capabilities and in the number and type of sensors included in their design. As the space of sensorimotor data available to a conventional or machine learning prosthetic control system increases in dimensionality and complexity, it becomes increasingly important that this data be represented in a useful and computationally efficient way. Well structured sensory data allows prosthetic control systems to make informed, appropriate control decisions. In this study, we explore the impact that increased sensorimotor information has on current machine learning prosthetic control approaches. Specifically, we examine the effect that high-dimensional sensory data has on the computation time and prediction performance of a true-online temporal-difference learning prediction method as embedded within a resource-limited upper-limb prosthesis control system. We present results comparing tile coding, the dominant linear representation for real-time prosthetic machine learning, with a newly proposed modification to Kanerva coding that we call selective Kanerva coding. In addition to showing promising results for selective Kanerva coding, our results confirm potential limitations to tile coding as the number of sensory input dimensions increases. To our knowledge, this study is the first to explicitly examine representations for realtime machine learning prosthetic devices in general terms. This work therefore provides an important step towards forming an efficient prosthesis-eye view of the world, wherein prompt and accurate representations of high-dimensional data may be provided to machine learning control systems within artificial limbs and other assistive rehabilitation technologies.

Air-Breathing Hypersonic Vehicle Tracking Control Based on Adaptive Dynamic Programming.

PubMed

Mu, Chaoxu; Ni, Zhen; Sun, Changyin; He, Haibo

2017-03-01

In this paper, we propose a data-driven supplementary control approach with adaptive learning capability for air-breathing hypersonic vehicle tracking control based on action-dependent heuristic dynamic programming (ADHDP). The control action is generated by the combination of sliding mode control (SMC) and the ADHDP controller to track the desired velocity and the desired altitude. In particular, the ADHDP controller observes the differences between the actual velocity/altitude and the desired velocity/altitude, and then provides a supplementary control action accordingly. The ADHDP controller does not rely on the accurate mathematical model function and is data driven. Meanwhile, it is capable to adjust its parameters online over time under various working conditions, which is very suitable for hypersonic vehicle system with parameter uncertainties and disturbances. We verify the adaptive supplementary control approach versus the traditional SMC in the cruising flight, and provide three simulation studies to illustrate the improved performance with the proposed approach.

Feedback Implementation of Zermelo's Optimal Control by Sugeno Approximation

NASA Technical Reports Server (NTRS)

Clifton, C.; Homaifax, A.; Bikdash, M.

1997-01-01

This paper proposes an approach to implement optimal control laws of nonlinear systems in real time. Our methodology does not require solving two-point boundary value problems online and may not require it off-line either. The optimal control law is learned using the original Sugeno controller (OSC) from a family of optimal trajectories. We compare the trajectories generated by the OSC and the trajectories yielded by the optimal feedback control law when applied to Zermelo's ship steering problem.

Predicting coronary artery disease using different artificial neural network models.

PubMed

Colak, M Cengiz; Colak, Cemil; Kocatürk, Hasan; Sağiroğlu, Seref; Barutçu, Irfan

2008-08-01

Eight different learning algorithms used for creating artificial neural network (ANN) models and the different ANN models in the prediction of coronary artery disease (CAD) are introduced. This work was carried out as a retrospective case-control study. Overall, 124 consecutive patients who had been diagnosed with CAD by coronary angiography (at least 1 coronary stenosis > 50% in major epicardial arteries) were enrolled in the work. Angiographically, the 113 people (group 2) with normal coronary arteries were taken as control subjects. Multi-layered perceptrons ANN architecture were applied. The ANN models trained with different learning algorithms were performed in 237 records, divided into training (n=171) and testing (n=66) data sets. The performance of prediction was evaluated by sensitivity, specificity and accuracy values based on standard definitions. The results have demonstrated that ANN models trained with eight different learning algorithms are promising because of high (greater than 71%) sensitivity, specificity and accuracy values in the prediction of CAD. Accuracy, sensitivity and specificity values varied between 83.63%-100%, 86.46%-100% and 74.67%-100% for training, respectively. For testing, the values were more than 71% for sensitivity, 76% for specificity and 81% for accuracy. It may be proposed that the use of different learning algorithms other than backpropagation and larger sample sizes can improve the performance of prediction. The proposed ANN models trained with these learning algorithms could be used a promising approach for predicting CAD without the need for invasive diagnostic methods and could help in the prognostic clinical decision.

The Effects of Rhythmicity and Amplitude on Transfer of Motor Learning

PubMed Central

Ben-Tov, Mor; Levy-Tzedek, Shelly; Karniel, Amir

2012-01-01

We perform rhythmic and discrete arm movements on a daily basis, yet the motor control literature is not conclusive regarding the mechanisms controlling these movements; does a single mechanism generate both movement types, or are they controlled by separate mechanisms? A recent study reported partial asymmetric transfer of learning from discrete movements to rhythmic movements. Other studies have shown transfer of learning between large-amplitude to small-amplitude movements. The goal of this study is to explore which aspect is important for learning to be transferred from one type of movement to another: rhythmicity, amplitude or both. We propose two hypotheses: (1) Rhythmic and discrete movements are generated by different mechanisms; therefore we expect to see a partial or no transfer of learning between the two types of movements; (2) Within each movement type (rhythmic/discrete), there will be asymmetric transition of learning from larger movements to smaller ones. We used a learning-transfer paradigm, in which 70 participants performed flexion/extension movements with their forearm, and switched between types of movement, which differed in amplitude and/or rhythmicity. We found partial transfer of learning between discrete and rhythmic movements, and an asymmetric transfer of learning from larger movements to smaller movements (within the same type of movement). Our findings suggest that there are two different mechanisms underlying the generation of rhythmic and discrete arm movements, and that practicing on larger movements helps perform smaller movements; the latter finding might have implications for rehabilitation. PMID:23056549

The effects of rhythmicity and amplitude on transfer of motor learning.

PubMed

Ben-Tov, Mor; Levy-Tzedek, Shelly; Karniel, Amir

2012-01-01

We perform rhythmic and discrete arm movements on a daily basis, yet the motor control literature is not conclusive regarding the mechanisms controlling these movements; does a single mechanism generate both movement types, or are they controlled by separate mechanisms? A recent study reported partial asymmetric transfer of learning from discrete movements to rhythmic movements. Other studies have shown transfer of learning between large-amplitude to small-amplitude movements. The goal of this study is to explore which aspect is important for learning to be transferred from one type of movement to another: rhythmicity, amplitude or both. We propose two hypotheses: (1) Rhythmic and discrete movements are generated by different mechanisms; therefore we expect to see a partial or no transfer of learning between the two types of movements; (2) Within each movement type (rhythmic/discrete), there will be asymmetric transition of learning from larger movements to smaller ones. We used a learning-transfer paradigm, in which 70 participants performed flexion/extension movements with their forearm, and switched between types of movement, which differed in amplitude and/or rhythmicity. We found partial transfer of learning between discrete and rhythmic movements, and an asymmetric transfer of learning from larger movements to smaller movements (within the same type of movement). Our findings suggest that there are two different mechanisms underlying the generation of rhythmic and discrete arm movements, and that practicing on larger movements helps perform smaller movements; the latter finding might have implications for rehabilitation.

Imagery May Arise from Associations Formed through Sensory Experience: A Network of Spiking Neurons Controlling a Robot Learns Visual Sequences in Order to Perform a Mental Rotation Task

PubMed Central

McKinstry, Jeffrey L.; Fleischer, Jason G.; Chen, Yanqing; Gall, W. Einar; Edelman, Gerald M.

2016-01-01

Mental imagery occurs “when a representation of the type created during the initial phases of perception is present but the stimulus is not actually being perceived.” How does the capability to perform mental imagery arise? Extending the idea that imagery arises from learned associations, we propose that mental rotation, a specific form of imagery, could arise through the mechanism of sequence learning–that is, by learning to regenerate the sequence of mental images perceived while passively observing a rotating object. To demonstrate the feasibility of this proposal, we constructed a simulated nervous system and embedded it within a behaving humanoid robot. By observing a rotating object, the system learns the sequence of neural activity patterns generated by the visual system in response to the object. After learning, it can internally regenerate a similar sequence of neural activations upon briefly viewing the static object. This system learns to perform a mental rotation task in which the subject must determine whether two objects are identical despite differences in orientation. As with human subjects, the time taken to respond is proportional to the angular difference between the two stimuli. Moreover, as reported in humans, the system fills in intermediate angles during the task, and this putative mental rotation activates the same pathways that are activated when the system views physical rotation. This work supports the proposal that mental rotation arises through sequence learning and the idea that mental imagery aids perception through learned associations, and suggests testable predictions for biological experiments. PMID:27653977

Intelligent power management in a vehicular system with multiple power sources

NASA Astrophysics Data System (ADS)

Murphey, Yi L.; Chen, ZhiHang; Kiliaris, Leonidas; Masrur, M. Abul

This paper presents an optimal online power management strategy applied to a vehicular power system that contains multiple power sources and deals with largely fluctuated load requests. The optimal online power management strategy is developed using machine learning and fuzzy logic. A machine learning algorithm has been developed to learn the knowledge about minimizing power loss in a Multiple Power Sources and Loads (M_PS&LD) system. The algorithm exploits the fact that different power sources used to deliver a load request have different power losses under different vehicle states. The machine learning algorithm is developed to train an intelligent power controller, an online fuzzy power controller, FPC_MPS, that has the capability of finding combinations of power sources that minimize power losses while satisfying a given set of system and component constraints during a drive cycle. The FPC_MPS was implemented in two simulated systems, a power system of four power sources, and a vehicle system of three power sources. Experimental results show that the proposed machine learning approach combined with fuzzy control is a promising technology for intelligent vehicle power management in a M_PS&LD power system.

Reinforcement interval type-2 fuzzy controller design by online rule generation and q-value-aided ant colony optimization.

PubMed

Juang, Chia-Feng; Hsu, Chia-Hung

2009-12-01

This paper proposes a new reinforcement-learning method using online rule generation and Q-value-aided ant colony optimization (ORGQACO) for fuzzy controller design. The fuzzy controller is based on an interval type-2 fuzzy system (IT2FS). The antecedent part in the designed IT2FS uses interval type-2 fuzzy sets to improve controller robustness to noise. There are initially no fuzzy rules in the IT2FS. The ORGQACO concurrently designs both the structure and parameters of an IT2FS. We propose an online interval type-2 rule generation method for the evolution of system structure and flexible partitioning of the input space. Consequent part parameters in an IT2FS are designed using Q -values and the reinforcement local-global ant colony optimization algorithm. This algorithm selects the consequent part from a set of candidate actions according to ant pheromone trails and Q-values, both of which are updated using reinforcement signals. The ORGQACO design method is applied to the following three control problems: 1) truck-backing control; 2) magnetic-levitation control; and 3) chaotic-system control. The ORGQACO is compared with other reinforcement-learning methods to verify its efficiency and effectiveness. Comparisons with type-1 fuzzy systems verify the noise robustness property of using an IT2FS.

Optimal teaching strategy in periodic impulsive knowledge dissemination system.

PubMed

Liu, Dan-Qing; Wu, Zhen-Qiang; Wang, Yu-Xin; Guo, Qiang; Liu, Jian-Guo

2017-01-01

Accurately describing the knowledge dissemination process is significant to enhance the performance of personalized education. In this study, considering the effect of periodic teaching activities on the learning process, we propose a periodic impulsive knowledge dissemination system to regenerate the knowledge dissemination process. Meanwhile, we put forward learning effectiveness which is an outcome of a trade-off between the benefits and costs raised by knowledge dissemination as objective function. Further, we investigate the optimal teaching strategy which can maximize learning effectiveness, to obtain the optimal effect of knowledge dissemination affected by the teaching activities. We solve this dynamic optimization problem by optimal control theory and get the optimization system. At last we numerically solve this system in several practical examples to make the conclusions intuitive and specific. The optimal teaching strategy proposed in this paper can be applied widely in the optimization problem of personal education and beneficial for enhancing the effect of knowledge dissemination.

Optimal teaching strategy in periodic impulsive knowledge dissemination system

PubMed Central

Liu, Dan-Qing; Wu, Zhen-Qiang; Wang, Yu-Xin; Guo, Qiang

2017-01-01

Accurately describing the knowledge dissemination process is significant to enhance the performance of personalized education. In this study, considering the effect of periodic teaching activities on the learning process, we propose a periodic impulsive knowledge dissemination system to regenerate the knowledge dissemination process. Meanwhile, we put forward learning effectiveness which is an outcome of a trade-off between the benefits and costs raised by knowledge dissemination as objective function. Further, we investigate the optimal teaching strategy which can maximize learning effectiveness, to obtain the optimal effect of knowledge dissemination affected by the teaching activities. We solve this dynamic optimization problem by optimal control theory and get the optimization system. At last we numerically solve this system in several practical examples to make the conclusions intuitive and specific. The optimal teaching strategy proposed in this paper can be applied widely in the optimization problem of personal education and beneficial for enhancing the effect of knowledge dissemination. PMID:28665961

A Car-Steering Model Based on an Adaptive Neuro-Fuzzy Controller

NASA Astrophysics Data System (ADS)

Amor, Mohamed Anis Ben; Oda, Takeshi; Watanabe, Shigeyoshi

This paper is concerned with the development of a car-steering model for traffic simulation. Our focus in this paper is to propose a model of the steering behavior of a human driver for different driving scenarios. These scenarios are modeled in a unified framework using the idea of target position. The proposed approach deals with the driver’s approximation and decision-making mechanisms in tracking a target position by means of fuzzy set theory. The main novelty in this paper lies in the development of a learning algorithm that has the intention to imitate the driver’s self-learning from his driving experience and to mimic his maneuvers on the steering wheel, using linear networks as local approximators in the corresponding fuzzy areas. Results obtained from the simulation of an obstacle avoidance scenario show the capability of the model to carry out a human-like behavior with emphasis on learned skills.

Improving Test-Taking Skills of LD Adolescents.

ERIC Educational Resources Information Center

Markel, Geraldine

1981-01-01

A multicomponent model to improve test taking skills of learning disabled (LD) adolescents is proposed to encourage anxiety management, problem solving skills, assertiveness, study skills, and student confidence and control. The role of the LD consultant in this process is described. (CL)

A specific implicit sequence learning deficit as an underlying cause of dyslexia? Investigating the role of attention in implicit learning tasks.

PubMed

Staels, Eva; Van den Broeck, Wim

2017-05-01

Recently, a general implicit sequence learning deficit was proposed as an underlying cause of dyslexia. This new hypothesis was investigated in the present study by including a number of methodological improvements, for example, the inclusion of appropriate control conditions. The second goal of the study was to explore the role of attentional functioning in implicit and explicit learning tasks. In a 2 × 2 within-subjects design 4 tasks were administered in 30 dyslexic and 38 control children: an implicit and explicit serial reaction time (RT) task and an implicit and explicit contextual cueing task. Attentional functioning was also administered. The entire learning curves of all tasks were analyzed using latent growth curve modeling in order to compare performances between groups and to examine the role of attentional functioning on the learning curves. The amount of implicit learning was similar for both groups. However, the dyslexic group showed slower RTs throughout the entire task. This group difference reduced and became nonsignificant after controlling for attentional functioning. Both implicit learning tasks, but none of the explicit learning tasks, were significantly affected by attentional functioning. Dyslexic children do not suffer from a specific implicit sequence learning deficit. The slower RTs of the dyslexic children throughout the entire implicit sequence learning process are caused by their comorbid attention problems and overall slowness. A key finding of the present study is that, in contrast to what was assumed for a long time, implicit learning relies on attentional resources, perhaps even more than explicit learning does. (PsycINFO Database Record (c) 2017 APA, all rights reserved).

Advanced Physiological Estimation of Cognitive Status

DTIC Science & Technology

2011-05-24

Research Triangle Park, NC 27709-2211 15. SUBJECT TERMS fatigue mental workload cognitive status EEG machine learning algorithms Leonard J. Trejo...Information Transfer (NOIT)” • ARO Proposal No. 56469-LS • Three-year basic research with UCLA team 2. “ EEG -guided Input Lateralization and Hemispheric...Activation with Neurofeedback for Display Data Control and Apprehension.” • ARO Proposal No. 59502-LS • One-year Infrastructure technology transfer to

An iterative learning strategy for the auto-tuning of the feedforward and feedback controller in type-1 diabetes.

PubMed

Fravolini, M L; Fabietti, P G

2014-01-01

This paper proposes a scheme for the control of the blood glucose in subjects with type-1 diabetes mellitus based on the subcutaneous (s.c.) glucose measurement and s.c. insulin administration. The tuning of the controller is based on an iterative learning strategy that exploits the repetitiveness of the daily feeding habit of a patient. The control consists of a mixed feedback and feedforward contribution whose parameters are tuned through an iterative learning process that is based on the day-by-day automated analysis of the glucose response to the infusion of exogenous insulin. The scheme does not require any a priori information on the patient insulin/glucose response, on the meal times and on the amount of ingested carbohydrates (CHOs). Thanks to the learning mechanism the scheme is able to improve its performance over time. A specific logic is also introduced for the detection and prevention of possible hypoglycaemia events. The effectiveness of the methodology has been validated using long-term simulation studies applied to a set of nine in silico patients considering realistic uncertainties on the meal times and on the quantities of ingested CHOs.

An adaptive learning control system for large flexible structures

NASA Technical Reports Server (NTRS)

Thau, F. E.

1985-01-01

The objective of the research has been to study the design of adaptive/learning control systems for the control of large flexible structures. In the first activity an adaptive/learning control methodology for flexible space structures was investigated. The approach was based on using a modal model of the flexible structure dynamics and an output-error identification scheme to identify modal parameters. In the second activity, a least-squares identification scheme was proposed for estimating both modal parameters and modal-to-actuator and modal-to-sensor shape functions. The technique was applied to experimental data obtained from the NASA Langley beam experiment. In the third activity, a separable nonlinear least-squares approach was developed for estimating the number of excited modes, shape functions, modal parameters, and modal amplitude and velocity time functions for a flexible structure. In the final research activity, a dual-adaptive control strategy was developed for regulating the modal dynamics and identifying modal parameters of a flexible structure. A min-max approach was used for finding an input to provide modal parameter identification while not exceeding reasonable bounds on modal displacement.

Adaptive optimal control of unknown constrained-input systems using policy iteration and neural networks.

PubMed

Modares, Hamidreza; Lewis, Frank L; Naghibi-Sistani, Mohammad-Bagher

2013-10-01

This paper presents an online policy iteration (PI) algorithm to learn the continuous-time optimal control solution for unknown constrained-input systems. The proposed PI algorithm is implemented on an actor-critic structure where two neural networks (NNs) are tuned online and simultaneously to generate the optimal bounded control policy. The requirement of complete knowledge of the system dynamics is obviated by employing a novel NN identifier in conjunction with the actor and critic NNs. It is shown how the identifier weights estimation error affects the convergence of the critic NN. A novel learning rule is developed to guarantee that the identifier weights converge to small neighborhoods of their ideal values exponentially fast. To provide an easy-to-check persistence of excitation condition, the experience replay technique is used. That is, recorded past experiences are used simultaneously with current data for the adaptation of the identifier weights. Stability of the whole system consisting of the actor, critic, system state, and system identifier is guaranteed while all three networks undergo adaptation. Convergence to a near-optimal control law is also shown. The effectiveness of the proposed method is illustrated with a simulation example.

Illusions of causality at the heart of pseudoscience.

PubMed

Matute, Helena; Yarritu, Ion; Vadillo, Miguel A

2011-08-01

Pseudoscience, superstitions, and quackery are serious problems that threaten public health and in which many variables are involved. Psychology, however, has much to say about them, as it is the illusory perceptions of causality of so many people that needs to be understood. The proposal we put forward is that these illusions arise from the normal functioning of the cognitive system when trying to associate causes and effects. Thus, we propose to apply basic research and theories on causal learning to reduce the impact of pseudoscience. We review the literature on the illusion of control and the causal learning traditions, and then present an experiment as an illustration of how this approach can provide fruitful ideas to reduce pseudoscientific thinking. The experiment first illustrates the development of a quackery illusion through the testimony of fictitious patients who report feeling better. Two different predictions arising from the integration of the causal learning and illusion of control domains are then proven effective in reducing this illusion. One is showing the testimony of people who feel better without having followed the treatment. The other is asking participants to think in causal terms rather than in terms of effectiveness. ©2010 The British Psychological Society.

An adaptive Hinfinity controller design for bank-to-turn missiles using ridge Gaussian neural networks.

PubMed

Lin, Chuan-Kai; Wang, Sheng-De

2004-11-01

A new autopilot design for bank-to-turn (BTT) missiles is presented. In the design of autopilot, a ridge Gaussian neural network with local learning capability and fewer tuning parameters than Gaussian neural networks is proposed to model the controlled nonlinear systems. We prove that the proposed ridge Gaussian neural network, which can be a universal approximator, equals the expansions of rotated and scaled Gaussian functions. Although ridge Gaussian neural networks can approximate the nonlinear and complex systems accurately, the small approximation errors may affect the tracking performance significantly. Therefore, by employing the Hinfinity control theory, it is easy to attenuate the effects of the approximation errors of the ridge Gaussian neural networks to a prescribed level. Computer simulation results confirm the effectiveness of the proposed ridge Gaussian neural networks-based autopilot with Hinfinity stabilization.

A Q-Learning Approach to Flocking With UAVs in a Stochastic Environment.

PubMed

Hung, Shao-Ming; Givigi, Sidney N

2017-01-01

In the past two decades, unmanned aerial vehicles (UAVs) have demonstrated their efficacy in supporting both military and civilian applications, where tasks can be dull, dirty, dangerous, or simply too costly with conventional methods. Many of the applications contain tasks that can be executed in parallel, hence the natural progression is to deploy multiple UAVs working together as a force multiplier. However, to do so requires autonomous coordination among the UAVs, similar to swarming behaviors seen in animals and insects. This paper looks at flocking with small fixed-wing UAVs in the context of a model-free reinforcement learning problem. In particular, Peng's Q(λ) with a variable learning rate is employed by the followers to learn a control policy that facilitates flocking in a leader-follower topology. The problem is structured as a Markov decision process, where the agents are modeled as small fixed-wing UAVs that experience stochasticity due to disturbances such as winds and control noises, as well as weight and balance issues. Learned policies are compared to ones solved using stochastic optimal control (i.e., dynamic programming) by evaluating the average cost incurred during flight according to a cost function. Simulation results demonstrate the feasibility of the proposed learning approach at enabling agents to learn how to flock in a leader-follower topology, while operating in a nonstationary stochastic environment.

Learning classifier systems for single and multiple mobile robots in unstructured environments

NASA Astrophysics Data System (ADS)

Bay, John S.

1995-12-01

The learning classifier system (LCS) is a learning production system that generates behavioral rules via an underlying discovery mechanism. The LCS architecture operates similarly to a blackboard architecture; i.e., by posted-message communications. But in the LCS, the message board is wiped clean at every time interval, thereby requiring no persistent shared resource. In this paper, we adapt the LCS to the problem of mobile robot navigation in completely unstructured environments. We consider the model of the robot itself, including its sensor and actuator structures, to be part of this environment, in addition to the world-model that includes a goal and obstacles at unknown locations. This requires a robot to learn its own I/O characteristics in addition to solving its navigation problem, but results in a learning controller that is equally applicable, unaltered, in robots with a wide variety of kinematic structures and sensing capabilities. We show the effectiveness of this LCS-based controller through both simulation and experimental trials with a small robot. We then propose a new architecture, the Distributed Learning Classifier System (DLCS), which generalizes the message-passing behavior of the LCS from internal messages within a single agent to broadcast massages among multiple agents. This communications mode requires little bandwidth and is easily implemented with inexpensive, off-the-shelf hardware. The DLCS is shown to have potential application as a learning controller for multiple intelligent agents.

Intelligent nonsingular terminal sliding-mode control using MIMO Elman neural network for piezo-flexural nanopositioning stage.

PubMed

Lin, Faa-Jeng; Lee, Shih-Yang; Chou, Po-Huan

2012-12-01

The objective of this study is to develop an intelligent nonsingular terminal sliding-mode control (INTSMC) system using an Elman neural network (ENN) for the threedimensional motion control of a piezo-flexural nanopositioning stage (PFNS). First, the dynamic model of the PFNS is derived in detail. Then, to achieve robust, accurate trajectory-tracking performance, a nonsingular terminal sliding-mode control (NTSMC) system is proposed for the tracking of the reference contours. The steady-state response of the control system can be improved effectively because of the addition of the nonsingularity in the NTSMC. Moreover, to relax the requirements of the bounds and discard the switching function in NTSMC, an INTSMC system using a multi-input-multioutput (MIMO) ENN estimator is proposed to improve the control performance and robustness of the PFNS. The ENN estimator is proposed to estimate the hysteresis phenomenon and lumped uncertainty, including the system parameters and external disturbance of the PFNS online. Furthermore, the adaptive learning algorithms for the training of the parameters of the ENN online are derived using the Lyapunov stability theorem. In addition, two robust compensators are proposed to confront the minimum reconstructed errors in INTSMC. Finally, some experimental results for the tracking of various contours are given to demonstrate the validity of the proposed INTSMC system for PFNS.

The hierarchical expert tuning of PID controllers using tools of soft computing.

PubMed

Karray, F; Gueaieb, W; Al-Sharhan, S

2002-01-01

We present soft computing-based results pertaining to the hierarchical tuning process of PID controllers located within the control loop of a class of nonlinear systems. The results are compared with PID controllers implemented either in a stand alone scheme or as a part of conventional gain scheduling structure. This work is motivated by the increasing need in the industry to design highly reliable and efficient controllers for dealing with regulation and tracking capabilities of complex processes characterized by nonlinearities and possibly time varying parameters. The soft computing-based controllers proposed are hybrid in nature in that they integrate within a well-defined hierarchical structure the benefits of hard algorithmic controllers with those having supervisory capabilities. The controllers proposed also have the distinct features of learning and auto-tuning without the need for tedious and computationally extensive online systems identification schemes.

An adaptive neuro fuzzy inference system controlled space cector pulse width modulation based HVDC light transmission system under AC fault conditions

NASA Astrophysics Data System (ADS)

Ajay Kumar, M.; Srikanth, N. V.

2014-03-01

In HVDC Light transmission systems, converter control is one of the major fields of present day research works. In this paper, fuzzy logic controller is utilized for controlling both the converters of the space vector pulse width modulation (SVPWM) based HVDC Light transmission systems. Due to its complexity in the rule base formation, an intelligent controller known as adaptive neuro fuzzy inference system (ANFIS) controller is also introduced in this paper. The proposed ANFIS controller changes the PI gains automatically for different operating conditions. A hybrid learning method which combines and exploits the best features of both the back propagation algorithm and least square estimation method is used to train the 5-layer ANFIS controller. The performance of the proposed ANFIS controller is compared and validated with the fuzzy logic controller and also with the fixed gain conventional PI controller. The simulations are carried out in the MATLAB/SIMULINK environment. The results reveal that the proposed ANFIS controller is reducing power fluctuations at both the converters. It also improves the dynamic performance of the test power system effectively when tested for various ac fault conditions.

Distributed Learning and Information Dynamics In Networked Autonomous Systems

DTIC Science & Technology

2015-11-20

2009 to June 30, 2015 4. TITLE AND SUBTITLE DISTRIBUTED LEARNING AND INFORMATION DYNAMICS IN NETWORKED AUTONOMOUS SYSTEMS 5a. CONTRACT NUMBER 5b...AUTONOMOUS SYSTEMS AFOSR Grant #FA9550–09–1–0538 PI: Eric Feron (current) Jeff S. Shamma (former) Georgia Institute of Technology Atlanta, GA 30332 1...Control. Design of event-based optimal remote estimation systems : We have proposed two new for- mulations to study the design of optimal remote

76 FR 65190 - Notice of Proposed Information Collection Requests

Federal Register 2010, 2011, 2012, 2013, 2014

2011-10-20

... Century Community Learning Centers Annual Performance Report. OMB Control Number: 1810-0668. Agency Form... et seq.), is to provide expanded academic enrichment opportunities for children attending low... authorization for the collection of data through Web-based, data-collection modules, the Annual Performance...

Emergence of Joint Attention through Bootstrap Learning based on the Mechanisms of Visual Attention and Learning with Self-evaluation

NASA Astrophysics Data System (ADS)

Nagai, Yukie; Hosoda, Koh; Morita, Akio; Asada, Minoru

This study argues how human infants acquire the ability of joint attention through interactions with their caregivers from a viewpoint of cognitive developmental robotics. In this paper, a mechanism by which a robot acquires sensorimotor coordination for joint attention through bootstrap learning is described. Bootstrap learning is a process by which a learner acquires higher capabilities through interactions with its environment based on embedded lower capabilities even if the learner does not receive any external evaluation nor the environment is controlled. The proposed mechanism for bootstrap learning of joint attention consists of the robot's embedded mechanisms: visual attention and learning with self-evaluation. The former is to find and attend to a salient object in the field of the robot's view, and the latter is to evaluate the success of visual attention, not joint attention, and then to learn the sensorimotor coordination. Since the object which the robot looks at based on visual attention does not always correspond to the object which the caregiver is looking at in an environment including multiple objects, the robot may have incorrect learning situations for joint attention as well as correct ones. However, the robot is expected to statistically lose the learning data of the incorrect ones as outliers because of its weaker correlation between the sensor input and the motor output than that of the correct ones, and consequently to acquire appropriate sensorimotor coordination for joint attention even if the caregiver does not provide any task evaluation to the robot. The experimental results show the validity of the proposed mechanism. It is suggested that the proposed mechanism could explain the developmental mechanism of infants' joint attention because the learning process of the robot's joint attention can be regarded as equivalent to the developmental process of infants' one.

Integration of Gravitational Torques in Cerebellar Pathways Allows for the Dynamic Inverse Computation of Vertical Pointing Movements of a Robot Arm

PubMed Central

Gentili, Rodolphe J.; Papaxanthis, Charalambos; Ebadzadeh, Mehdi; Eskiizmirliler, Selim; Ouanezar, Sofiane; Darlot, Christian

2009-01-01

Background Several authors suggested that gravitational forces are centrally represented in the brain for planning, control and sensorimotor predictions of movements. Furthermore, some studies proposed that the cerebellum computes the inverse dynamics (internal inverse model) whereas others suggested that it computes sensorimotor predictions (internal forward model). Methodology/Principal Findings This study proposes a model of cerebellar pathways deduced from both biological and physical constraints. The model learns the dynamic inverse computation of the effect of gravitational torques from its sensorimotor predictions without calculating an explicit inverse computation. By using supervised learning, this model learns to control an anthropomorphic robot arm actuated by two antagonists McKibben artificial muscles. This was achieved by using internal parallel feedback loops containing neural networks which anticipate the sensorimotor consequences of the neural commands. The artificial neural networks architecture was similar to the large-scale connectivity of the cerebellar cortex. Movements in the sagittal plane were performed during three sessions combining different initial positions, amplitudes and directions of movements to vary the effects of the gravitational torques applied to the robotic arm. The results show that this model acquired an internal representation of the gravitational effects during vertical arm pointing movements. Conclusions/Significance This is consistent with the proposal that the cerebellar cortex contains an internal representation of gravitational torques which is encoded through a learning process. Furthermore, this model suggests that the cerebellum performs the inverse dynamics computation based on sensorimotor predictions. This highlights the importance of sensorimotor predictions of gravitational torques acting on upper limb movements performed in the gravitational field. PMID:19384420

Improving the efficiency of dissolved oxygen control using an on-line control system based on a genetic algorithm evolving FWNN software sensor.

PubMed

Ruan, Jujun; Zhang, Chao; Li, Ya; Li, Peiyi; Yang, Zaizhi; Chen, Xiaohong; Huang, Mingzhi; Zhang, Tao

2017-02-01

This work proposes an on-line hybrid intelligent control system based on a genetic algorithm (GA) evolving fuzzy wavelet neural network software sensor to control dissolved oxygen (DO) in an anaerobic/anoxic/oxic process for treating papermaking wastewater. With the self-learning and memory abilities of neural network, handling the uncertainty capacity of fuzzy logic, analyzing local detail superiority of wavelet transform and global search of GA, this proposed control system can extract the dynamic behavior and complex interrelationships between various operation variables. The results indicate that the reasonable forecasting and control performances were achieved with optimal DO, and the effluent quality was stable at and below the desired values in real time. Our proposed hybrid approach proved to be a robust and effective DO control tool, attaining not only adequate effluent quality but also minimizing the demand for energy, and is easily integrated into a global monitoring system for purposes of cost management. Copyright © 2016 Elsevier Ltd. All rights reserved.

Dopamine, reward learning, and active inference

PubMed Central

FitzGerald, Thomas H. B.; Dolan, Raymond J.; Friston, Karl

2015-01-01

Temporal difference learning models propose phasic dopamine signaling encodes reward prediction errors that drive learning. This is supported by studies where optogenetic stimulation of dopamine neurons can stand in lieu of actual reward. Nevertheless, a large body of data also shows that dopamine is not necessary for learning, and that dopamine depletion primarily affects task performance. We offer a resolution to this paradox based on an hypothesis that dopamine encodes the precision of beliefs about alternative actions, and thus controls the outcome-sensitivity of behavior. We extend an active inference scheme for solving Markov decision processes to include learning, and show that simulated dopamine dynamics strongly resemble those actually observed during instrumental conditioning. Furthermore, simulated dopamine depletion impairs performance but spares learning, while simulated excitation of dopamine neurons drives reward learning, through aberrant inference about outcome states. Our formal approach provides a novel and parsimonious reconciliation of apparently divergent experimental findings. PMID:26581305

Dopamine, reward learning, and active inference.

PubMed

FitzGerald, Thomas H B; Dolan, Raymond J; Friston, Karl

2015-01-01

Temporal difference learning models propose phasic dopamine signaling encodes reward prediction errors that drive learning. This is supported by studies where optogenetic stimulation of dopamine neurons can stand in lieu of actual reward. Nevertheless, a large body of data also shows that dopamine is not necessary for learning, and that dopamine depletion primarily affects task performance. We offer a resolution to this paradox based on an hypothesis that dopamine encodes the precision of beliefs about alternative actions, and thus controls the outcome-sensitivity of behavior. We extend an active inference scheme for solving Markov decision processes to include learning, and show that simulated dopamine dynamics strongly resemble those actually observed during instrumental conditioning. Furthermore, simulated dopamine depletion impairs performance but spares learning, while simulated excitation of dopamine neurons drives reward learning, through aberrant inference about outcome states. Our formal approach provides a novel and parsimonious reconciliation of apparently divergent experimental findings.

Koopman Operator Framework for Time Series Modeling and Analysis

NASA Astrophysics Data System (ADS)

Surana, Amit

2018-01-01

We propose an interdisciplinary framework for time series classification, forecasting, and anomaly detection by combining concepts from Koopman operator theory, machine learning, and linear systems and control theory. At the core of this framework is nonlinear dynamic generative modeling of time series using the Koopman operator which is an infinite-dimensional but linear operator. Rather than working with the underlying nonlinear model, we propose two simpler linear representations or model forms based on Koopman spectral properties. We show that these model forms are invariants of the generative model and can be readily identified directly from data using techniques for computing Koopman spectral properties without requiring the explicit knowledge of the generative model. We also introduce different notions of distances on the space of such model forms which is essential for model comparison/clustering. We employ the space of Koopman model forms equipped with distance in conjunction with classical machine learning techniques to develop a framework for automatic feature generation for time series classification. The forecasting/anomaly detection framework is based on using Koopman model forms along with classical linear systems and control approaches. We demonstrate the proposed framework for human activity classification, and for time series forecasting/anomaly detection in power grid application.

Behavior generation strategy of artificial behavioral system by self-learning paradigm for autonomous robot tasks

NASA Astrophysics Data System (ADS)

Dağlarli, Evren; Temeltaş, Hakan

2008-04-01

In this study, behavior generation and self-learning paradigms are investigated for the real-time applications of multi-goal mobile robot tasks. The method is capable to generate new behaviors and it combines them in order to achieve multi goal tasks. The proposed method is composed from three layers: Behavior Generating Module, Coordination Level and Emotion -Motivation Level. Last two levels use Hidden Markov models to manage dynamical structure of behaviors. The kinematics and dynamic model of the mobile robot with non-holonomic constraints are considered in the behavior based control architecture. The proposed method is tested on a four-wheel driven and four-wheel steered mobile robot with constraints in simulation environment and results are obtained successfully.

Dissecting the mechanisms of squirrel monkey (Saimiri boliviensis) social learning

PubMed Central

Holmes, AN; Williams, LE; Brosnan, SF

2013-01-01

Although the social learning abilities of monkeys have been well documented, this research has only focused on a few species. Furthermore, of those that also incorporated dissections of social learning mechanisms, the majority studied either capuchins (Cebus apella) or marmosets (Callithrix jacchus). To gain a broader understanding of how monkeys gain new skills, we tested squirrel monkeys (Saimiri boliviensis) which have never been studied in tests of social learning mechanisms. To determine whether S. boliviensis can socially learn, we ran “open diffusion” tests with monkeys housed in two social groups (N = 23). Over the course of 10 20-min sessions, the monkeys in each group observed a trained group member retrieving a mealworm from a bidirectional task (the “Slide-box”). Two thirds (67%) of these monkeys both learned how to operate the Slide-box and they also moved the door significantly more times in the direction modeled by the trained demonstrator than the alternative direction. To tease apart the underlying social learning mechanisms we ran a series of three control conditions with 35 squirrel monkeys that had no previous experience with the Slide-box. The first replicated the experimental open diffusion sessions but without the inclusion of a trained model, the second was a no-information control with dyads of monkeys, and the third was a ‘ghost’ display shown to individual monkeys. The first two controls tested for the importance of social support (mere presence effect) and the ghost display showed the affordances of the task to the monkeys. The monkeys showed a certain level of success in the group control (54% of subjects solved the task on one or more occasions) and paired controls (28% were successful) but none were successful in the ghost control. We propose that the squirrel monkeys’ learning, observed in the experimental open diffusion tests, can be best described by a combination of social learning mechanisms in concert; in this case, those mechanisms are most likely object movement reenactment and social facilitation. We discuss the interplay of these mechanisms and how they related to learning shown by other primate species. PMID:23638347

Data Randomization and Cluster-Based Partitioning for Botnet Intrusion Detection.

PubMed

Al-Jarrah, Omar Y; Alhussein, Omar; Yoo, Paul D; Muhaidat, Sami; Taha, Kamal; Kim, Kwangjo

2016-08-01

Botnets, which consist of remotely controlled compromised machines called bots, provide a distributed platform for several threats against cyber world entities and enterprises. Intrusion detection system (IDS) provides an efficient countermeasure against botnets. It continually monitors and analyzes network traffic for potential vulnerabilities and possible existence of active attacks. A payload-inspection-based IDS (PI-IDS) identifies active intrusion attempts by inspecting transmission control protocol and user datagram protocol packet's payload and comparing it with previously seen attacks signatures. However, the PI-IDS abilities to detect intrusions might be incapacitated by packet encryption. Traffic-based IDS (T-IDS) alleviates the shortcomings of PI-IDS, as it does not inspect packet payload; however, it analyzes packet header to identify intrusions. As the network's traffic grows rapidly, not only the detection-rate is critical, but also the efficiency and the scalability of IDS become more significant. In this paper, we propose a state-of-the-art T-IDS built on a novel randomized data partitioned learning model (RDPLM), relying on a compact network feature set and feature selection techniques, simplified subspacing and a multiple randomized meta-learning technique. The proposed model has achieved 99.984% accuracy and 21.38 s training time on a well-known benchmark botnet dataset. Experiment results demonstrate that the proposed methodology outperforms other well-known machine-learning models used in the same detection task, namely, sequential minimal optimization, deep neural network, C4.5, reduced error pruning tree, and randomTree.

Toward experimental validation of a model for human sensorimotor learning and control in teleoperation

NASA Astrophysics Data System (ADS)

Roth, Eatai; Howell, Darrin; Beckwith, Cydney; Burden, Samuel A.

2017-05-01

Humans, interacting with cyber-physical systems (CPS), formulate beliefs about the system's dynamics. It is natural to expect that human operators, tasked with teleoperation, use these beliefs to control the remote robot. For tracking tasks in the resulting human-cyber-physical system (HCPS), theory suggests that human operators can achieve exponential tracking (in stable systems) without state estimation provided they possess an accurate model of the system's dynamics. This internalized inverse model, however, renders a portion of the system state unobservable to the human operator—the zero dynamics. Prior work shows humans can track through observable linear dynamics, thus we focus on nonlinear dynamics rendered unobservable through tracking control. We propose experiments to assess the human operator's ability to learn and invert such models, and distinguish this behavior from that achieved by pure feedback control.

Development of a neuromorphic control system for a lightweight humanoid robot

NASA Astrophysics Data System (ADS)

Folgheraiter, Michele; Keldibek, Amina; Aubakir, Bauyrzhan; Salakchinov, Shyngys; Gini, Giuseppina; Mauro Franchi, Alessio; Bana, Matteo

2017-03-01

A neuromorphic control system for a lightweight middle size humanoid biped robot built using 3D printing techniques is proposed. The control architecture consists of different modules capable to learn and autonomously reproduce complex periodic trajectories. Each module is represented by a chaotic Recurrent Neural Network (RNN) with a core of dynamic neurons randomly and sparsely connected with fixed synapses. A set of read-out units with adaptable synapses realize a linear combination of the neurons output in order to reproduce the target signals. Different experiments were conducted to find out the optimal initialization for the RNN’s parameters. From simulation results, using normalized signals obtained from the robot model, it was proven that all the instances of the control module can learn and reproduce the target trajectories with an average RMS error of 1.63 and variance 0.74.

Open-closed-loop iterative learning control for a class of nonlinear systems with random data dropouts

NASA Astrophysics Data System (ADS)

Cheng, X. Y.; Wang, H. B.; Jia, Y. L.; Dong, YH

2018-05-01

In this paper, an open-closed-loop iterative learning control (ILC) algorithm is constructed for a class of nonlinear systems subjecting to random data dropouts. The ILC algorithm is implemented by a networked control system (NCS), where only the off-line data is transmitted by network while the real-time data is delivered in the point-to-point way. Thus, there are two controllers rather than one in the control system, which makes better use of the saved and current information and thereby improves the performance achieved by open-loop control alone. During the transfer of off-line data between the nonlinear plant and the remote controller data dropout occurs randomly and the data dropout rate is modeled as a binary Bernoulli random variable. Both measurement and control data dropouts are taken into consideration simultaneously. The convergence criterion is derived based on rigorous analysis. Finally, the simulation results verify the effectiveness of the proposed method.

Data-Driven Robust M-LS-SVR-Based NARX Modeling for Estimation and Control of Molten Iron Quality Indices in Blast Furnace Ironmaking.

PubMed

Zhou, Ping; Guo, Dongwei; Wang, Hong; Chai, Tianyou

2017-09-29

Optimal operation of an industrial blast furnace (BF) ironmaking process largely depends on a reliable measurement of molten iron quality (MIQ) indices, which are not feasible using the conventional sensors. This paper proposes a novel data-driven robust modeling method for the online estimation and control of MIQ indices. First, a nonlinear autoregressive exogenous (NARX) model is constructed for the MIQ indices to completely capture the nonlinear dynamics of the BF process. Then, considering that the standard least-squares support vector regression (LS-SVR) cannot directly cope with the multioutput problem, a multitask transfer learning is proposed to design a novel multioutput LS-SVR (M-LS-SVR) for the learning of the NARX model. Furthermore, a novel M-estimator is proposed to reduce the interference of outliers and improve the robustness of the M-LS-SVR model. Since the weights of different outlier data are properly given by the weight function, their corresponding contributions on modeling can properly be distinguished, thus a robust modeling result can be achieved. Finally, a novel multiobjective evaluation index on the modeling performance is developed by comprehensively considering the root-mean-square error of modeling and the correlation coefficient on trend fitting, based on which the nondominated sorting genetic algorithm II is used to globally optimize the model parameters. Both experiments using industrial data and industrial applications illustrate that the proposed method can eliminate the adverse effect caused by the fluctuation of data in BF process efficiently. This indicates its stronger robustness and higher accuracy. Moreover, control testing shows that the developed model can be well applied to realize data-driven control of the BF process.

Data-Driven Robust M-LS-SVR-Based NARX Modeling for Estimation and Control of Molten Iron Quality Indices in Blast Furnace Ironmaking

DOE Office of Scientific and Technical Information (OSTI.GOV)

Zhou, Ping; Guo, Dongwei; Wang, Hong

Optimal operation of an industrial blast furnace (BF) ironmaking process largely depends on a reliable measurement of molten iron quality (MIQ) indices, which are not feasible using the conventional sensors. This paper proposes a novel data-driven robust modeling method for the online estimation and control of MIQ indices. First, a nonlinear autoregressive exogenous (NARX) model is constructed for the MIQ indices to completely capture the nonlinear dynamics of the BF process. Then, considering that the standard least-squares support vector regression (LS-SVR) cannot directly cope with the multioutput problem, a multitask transfer learning is proposed to design a novel multioutput LS-SVRmore » (M-LS-SVR) for the learning of the NARX model. Furthermore, a novel M-estimator is proposed to reduce the interference of outliers and improve the robustness of the M-LS-SVR model. Since the weights of different outlier data are properly given by the weight function, their corresponding contributions on modeling can properly be distinguished, thus a robust modeling result can be achieved. Finally, a novel multiobjective evaluation index on the modeling performance is developed by comprehensively considering the root-mean-square error of modeling and the correlation coefficient on trend fitting, based on which the nondominated sorting genetic algorithm II is used to globally optimize the model parameters. Both experiments using industrial data and industrial applications illustrate that the proposed method can eliminate the adverse effect caused by the fluctuation of data in BF process efficiently. In conclusion, this indicates its stronger robustness and higher accuracy. Moreover, control testing shows that the developed model can be well applied to realize data-driven control of the BF process.« less

Data-Driven Robust M-LS-SVR-Based NARX Modeling for Estimation and Control of Molten Iron Quality Indices in Blast Furnace Ironmaking

DOE PAGES

Zhou, Ping; Guo, Dongwei; Wang, Hong; ...

2017-09-29

Optimal operation of an industrial blast furnace (BF) ironmaking process largely depends on a reliable measurement of molten iron quality (MIQ) indices, which are not feasible using the conventional sensors. This paper proposes a novel data-driven robust modeling method for the online estimation and control of MIQ indices. First, a nonlinear autoregressive exogenous (NARX) model is constructed for the MIQ indices to completely capture the nonlinear dynamics of the BF process. Then, considering that the standard least-squares support vector regression (LS-SVR) cannot directly cope with the multioutput problem, a multitask transfer learning is proposed to design a novel multioutput LS-SVRmore » (M-LS-SVR) for the learning of the NARX model. Furthermore, a novel M-estimator is proposed to reduce the interference of outliers and improve the robustness of the M-LS-SVR model. Since the weights of different outlier data are properly given by the weight function, their corresponding contributions on modeling can properly be distinguished, thus a robust modeling result can be achieved. Finally, a novel multiobjective evaluation index on the modeling performance is developed by comprehensively considering the root-mean-square error of modeling and the correlation coefficient on trend fitting, based on which the nondominated sorting genetic algorithm II is used to globally optimize the model parameters. Both experiments using industrial data and industrial applications illustrate that the proposed method can eliminate the adverse effect caused by the fluctuation of data in BF process efficiently. In conclusion, this indicates its stronger robustness and higher accuracy. Moreover, control testing shows that the developed model can be well applied to realize data-driven control of the BF process.« less

Divide et impera: subgoaling reduces the complexity of probabilistic inference and problem solving.

PubMed

Maisto, Domenico; Donnarumma, Francesco; Pezzulo, Giovanni

2015-03-06

It has long been recognized that humans (and possibly other animals) usually break problems down into smaller and more manageable problems using subgoals. Despite a general consensus that subgoaling helps problem solving, it is still unclear what the mechanisms guiding online subgoal selection are during the solution of novel problems for which predefined solutions are not available. Under which conditions does subgoaling lead to optimal behaviour? When is subgoaling better than solving a problem from start to finish? Which is the best number and sequence of subgoals to solve a given problem? How are these subgoals selected during online inference? Here, we present a computational account of subgoaling in problem solving. Following Occam's razor, we propose that good subgoals are those that permit planning solutions and controlling behaviour using less information resources, thus yielding parsimony in inference and control. We implement this principle using approximate probabilistic inference: subgoals are selected using a sampling method that considers the descriptive complexity of the resulting sub-problems. We validate the proposed method using a standard reinforcement learning benchmark (four-rooms scenario) and show that the proposed method requires less inferential steps and permits selecting more compact control programs compared to an equivalent procedure without subgoaling. Furthermore, we show that the proposed method offers a mechanistic explanation of the neuronal dynamics found in the prefrontal cortex of monkeys that solve planning problems. Our computational framework provides a novel integrative perspective on subgoaling and its adaptive advantages for planning, control and learning, such as for example lowering cognitive effort and working memory load. © 2015 The Author(s) Published by the Royal Society. All rights reserved.

On the Effectiveness of a Neural Network for Adaptive External Pacing.

ERIC Educational Resources Information Center

Montazemi, Ali R.; Wang, Feng

1995-01-01

Proposes a neural network model for an intelligent tutoring system featuring adaptive external control of student pacing. An experiment was conducted, and students using adaptive external pacing experienced improved mastery learning and increased motivation for time management. Contains 66 references. (JKP)

Robust model predictive control of nonlinear systems with unmodeled dynamics and bounded uncertainties based on neural networks.

PubMed

Yan, Zheng; Wang, Jun

2014-03-01

This paper presents a neural network approach to robust model predictive control (MPC) for constrained discrete-time nonlinear systems with unmodeled dynamics affected by bounded uncertainties. The exact nonlinear model of underlying process is not precisely known, but a partially known nominal model is available. This partially known nonlinear model is first decomposed to an affine term plus an unknown high-order term via Jacobian linearization. The linearization residue combined with unmodeled dynamics is then modeled using an extreme learning machine via supervised learning. The minimax methodology is exploited to deal with bounded uncertainties. The minimax optimization problem is reformulated as a convex minimization problem and is iteratively solved by a two-layer recurrent neural network. The proposed neurodynamic approach to nonlinear MPC improves the computational efficiency and sheds a light for real-time implementability of MPC technology. Simulation results are provided to substantiate the effectiveness and characteristics of the proposed approach.

Information-theoretic approach to interactive learning

NASA Astrophysics Data System (ADS)

Still, S.

2009-01-01

The principles of statistical mechanics and information theory play an important role in learning and have inspired both theory and the design of numerous machine learning algorithms. The new aspect in this paper is a focus on integrating feedback from the learner. A quantitative approach to interactive learning and adaptive behavior is proposed, integrating model- and decision-making into one theoretical framework. This paper follows simple principles by requiring that the observer's world model and action policy should result in maximal predictive power at minimal complexity. Classes of optimal action policies and of optimal models are derived from an objective function that reflects this trade-off between prediction and complexity. The resulting optimal models then summarize, at different levels of abstraction, the process's causal organization in the presence of the learner's actions. A fundamental consequence of the proposed principle is that the learner's optimal action policies balance exploration and control as an emerging property. Interestingly, the explorative component is present in the absence of policy randomness, i.e. in the optimal deterministic behavior. This is a direct result of requiring maximal predictive power in the presence of feedback.

Role of the cerebellum in reaching movements in humans. II. A neural model of the intermediate cerebellum.

PubMed

Schweighofer, N; Spoelstra, J; Arbib, M A; Kawato, M

1998-01-01

The cerebellum is essential for the control of multijoint movements; when the cerebellum is lesioned, the performance error is more than the summed errors produced by single joints. In the companion paper (Schweighofer et al., 1998), a functional anatomical model for visually guided arm movement was proposed. The model comprised a basic feedforward/feedback controller with realistic transmission delays and was connected to a two-link, six-muscle, planar arm. In the present study, we examined the role of the cerebellum in reaching movements by embedding a novel, detailed cerebellar neural network in this functional control model. We could derive realistic cerebellar inputs and the role of the cerebellum in learning to control the arm was assessed. This cerebellar network learned the part of the inverse dynamics of the arm not provided by the basic feedforward/feedback controller. Despite realistically low inferior olive firing rates and noisy mossy fibre inputs, the model could reduce the error between intended and planned movements. The responses of the different cell groups were comparable to those of biological cell groups. In particular, the modelled Purkinje cells exhibited directional tuning after learning and the parallel fibres, due to their length, provide Purkinje cells with the input required for this coordination task. The inferior olive responses contained two different components; the earlier response, locked to movement onset, was always present and the later response disappeared after learning. These results support the theory that the cerebellum is involved in motor learning.

Sensori-Motor Learning with Movement Sonification: Perspectives from Recent Interdisciplinary Studies.

PubMed

Bevilacqua, Frédéric; Boyer, Eric O; Françoise, Jules; Houix, Olivier; Susini, Patrick; Roby-Brami, Agnès; Hanneton, Sylvain

2016-01-01

This article reports on an interdisciplinary research project on movement sonification for sensori-motor learning. First, we describe different research fields which have contributed to movement sonification, from music technology including gesture-controlled sound synthesis, sonic interaction design, to research on sensori-motor learning with auditory-feedback. In particular, we propose to distinguish between sound-oriented tasks and movement-oriented tasks in experiments involving interactive sound feedback. We describe several research questions and recently published results on movement control, learning and perception. In particular, we studied the effect of the auditory feedback on movements considering several cases: from experiments on pointing and visuo-motor tracking to more complex tasks where interactive sound feedback can guide movements, or cases of sensory substitution where the auditory feedback can inform on object shapes. We also developed specific methodologies and technologies for designing the sonic feedback and movement sonification. We conclude with a discussion on key future research challenges in sensori-motor learning with movement sonification. We also point out toward promising applications such as rehabilitation, sport training or product design.

Verification hybrid control of a wheeled mobile robot and manipulator

NASA Astrophysics Data System (ADS)

Muszynska, Magdalena; Burghardt, Andrzej; Kurc, Krzysztof; Szybicki, Dariusz

2016-04-01

In this article, innovative approaches to realization of the wheeled mobile robots and manipulator tracking are presented. Conceptions include application of the neural-fuzzy systems to compensation of the controlled system's nonlinearities in the tracking control task. Proposed control algorithms work on-line, contain structure, that adapt to the changeable work conditions of the controlled systems, and do not require the preliminary learning. The algorithm was verification on the real object which was a Scorbot - ER 4pc robotic manipulator and a Pioneer - 2DX mobile robot.

Implementation of real-time energy management strategy based on reinforcement learning for hybrid electric vehicles and simulation validation

PubMed Central

Kong, Zehui; Liu, Teng

2017-01-01

To further improve the fuel economy of series hybrid electric tracked vehicles, a reinforcement learning (RL)-based real-time energy management strategy is developed in this paper. In order to utilize the statistical characteristics of online driving schedule effectively, a recursive algorithm for the transition probability matrix (TPM) of power-request is derived. The reinforcement learning (RL) is applied to calculate and update the control policy at regular time, adapting to the varying driving conditions. A facing-forward powertrain model is built in detail, including the engine-generator model, battery model and vehicle dynamical model. The robustness and adaptability of real-time energy management strategy are validated through the comparison with the stationary control strategy based on initial transition probability matrix (TPM) generated from a long naturalistic driving cycle in the simulation. Results indicate that proposed method has better fuel economy than stationary one and is more effective in real-time control. PMID:28671967

Implementation of real-time energy management strategy based on reinforcement learning for hybrid electric vehicles and simulation validation.

PubMed

Kong, Zehui; Zou, Yuan; Liu, Teng

2017-01-01

To further improve the fuel economy of series hybrid electric tracked vehicles, a reinforcement learning (RL)-based real-time energy management strategy is developed in this paper. In order to utilize the statistical characteristics of online driving schedule effectively, a recursive algorithm for the transition probability matrix (TPM) of power-request is derived. The reinforcement learning (RL) is applied to calculate and update the control policy at regular time, adapting to the varying driving conditions. A facing-forward powertrain model is built in detail, including the engine-generator model, battery model and vehicle dynamical model. The robustness and adaptability of real-time energy management strategy are validated through the comparison with the stationary control strategy based on initial transition probability matrix (TPM) generated from a long naturalistic driving cycle in the simulation. Results indicate that proposed method has better fuel economy than stationary one and is more effective in real-time control.

An improved adaptive control for repetitive motion of robots

NASA Technical Reports Server (NTRS)

Pourboghrat, F.

1989-01-01

An adaptive control algorithm is proposed for a class of nonlinear systems, such as robotic manipulators, which is capable of improving its performance in repetitive motions. When the task is repeated, the error between the desired trajectory and that of the system is guaranteed to decrease. The design is based on the combination of a direct adaptive control and a learning process. This method does not require any knowledge of the dynamic parameters of the system.

The race to learn: spike timing and STDP can coordinate learning and recall in CA3.

PubMed

Nolan, Christopher R; Wyeth, Gordon; Milford, Michael; Wiles, Janet

2011-06-01

The CA3 region of the hippocampus has long been proposed as an autoassociative network performing pattern completion on known inputs. The dentate gyrus (DG) region is often proposed as a network performing the complementary function of pattern separation. Neural models of pattern completion and separation generally designate explicit learning phases to encode new information and assume an ideal fixed threshold at which to stop learning new patterns and begin recalling known patterns. Memory systems are significantly more complex in practice, with the degree of memory recall depending on context-specific goals. Here, we present our spike-timing separation and completion (STSC) model of the entorhinal cortex (EC), DG, and CA3 network, ascribing to each region a role similar to that in existing models but adding a temporal dimension by using a spiking neural network. Simulation results demonstrate that (a) spike-timing dependent plasticity in the EC-CA3 synapses provides a pattern completion ability without recurrent CA3 connections, (b) the race between activation of CA3 cells via EC-CA3 synapses and activation of the same cells via DG-CA3 synapses distinguishes novel from known inputs, and (c) modulation of the EC-CA3 synapses adjusts the learned versus test input similarity required to evoke a direct CA3 response prior to any DG activity, thereby adjusting the pattern completion threshold. These mechanisms suggest that spike timing can arbitrate between learning and recall based on the novelty of each individual input, ensuring control of the learn-recall decision resides in the same subsystem as the learned memories themselves. The proposed modulatory signal does not override this decision but biases the system toward either learning or recall. The model provides an explanation for empirical observations that a reduction in novelty produces a corresponding reduction in the latency of responses in CA3 and CA1. Copyright © 2010 Wiley-Liss, Inc.

Four-chamber heart modeling and automatic segmentation for 3-D cardiac CT volumes using marginal space learning and steerable features.

PubMed

Zheng, Yefeng; Barbu, Adrian; Georgescu, Bogdan; Scheuering, Michael; Comaniciu, Dorin

2008-11-01

We propose an automatic four-chamber heart segmentation system for the quantitative functional analysis of the heart from cardiac computed tomography (CT) volumes. Two topics are discussed: heart modeling and automatic model fitting to an unseen volume. Heart modeling is a nontrivial task since the heart is a complex nonrigid organ. The model must be anatomically accurate, allow manual editing, and provide sufficient information to guide automatic detection and segmentation. Unlike previous work, we explicitly represent important landmarks (such as the valves and the ventricular septum cusps) among the control points of the model. The control points can be detected reliably to guide the automatic model fitting process. Using this model, we develop an efficient and robust approach for automatic heart chamber segmentation in 3-D CT volumes. We formulate the segmentation as a two-step learning problem: anatomical structure localization and boundary delineation. In both steps, we exploit the recent advances in learning discriminative models. A novel algorithm, marginal space learning (MSL), is introduced to solve the 9-D similarity transformation search problem for localizing the heart chambers. After determining the pose of the heart chambers, we estimate the 3-D shape through learning-based boundary delineation. The proposed method has been extensively tested on the largest dataset (with 323 volumes from 137 patients) ever reported in the literature. To the best of our knowledge, our system is the fastest with a speed of 4.0 s per volume (on a dual-core 3.2-GHz processor) for the automatic segmentation of all four chambers.

Rasch measurement of self-regulated learning in an information and communication technology (ICT)-rich environment.

PubMed

Njiru, Joseph N; Waugh, Russell F

2007-01-01

This report describes how a linear scale of self-regulated learning in an ICT-rich environment was created by analysing student data using the Rasch measurement model. A person convenience sample of (N = 409) university students in Western Australia was used. The stem-item sample was initially 41, answered in two perspectives ("I aim for this" and "I actually do this"), and reduced to 16 that fitted the measurement model to form a unidimensional scale. Items for motivation (extrinsic rewards, intrinsic rewards, and social rewards), academic goals (fear of performing poorly) (but not standards), self-learning beliefs (ability and interest), task management (strategies and time management) (but not cooperative learning), Volition (action control (but not environmental control), and self-evaluation (cognitive self-evaluation and metacognition) fitted the measurement model. The proportion of observed variance considered true was 0.90. A new instrument is proposed to handle the conceptually valid but non-fitting items. Characteristics of high self-regulated learners are measured.

Learning Based Bidding Strategy for HVAC Systems in Double Auction Retail Energy Markets

DOE Office of Scientific and Technical Information (OSTI.GOV)

Sun, Yannan; Somani, Abhishek; Carroll, Thomas E.

In this paper, a bidding strategy is proposed using reinforcement learning for HVAC systems in a double auction market. The bidding strategy does not require a specific model-based representation of behavior, i.e., a functional form to translate indoor house temperatures into bid prices. The results from reinforcement learning based approach are compared with the HVAC bidding approach used in the AEP gridSMART® smart grid demonstration project and it is shown that the model-free (learning based) approach tracks well the results from the model-based behavior. Successful use of model-free approaches to represent device-level economic behavior may help develop similar approaches tomore » represent behavior of more complex devices or groups of diverse devices, such as in a building. Distributed control requires an understanding of decision making processes of intelligent agents so that appropriate mechanisms may be developed to control and coordinate their responses, and model-free approaches to represent behavior will be extremely useful in that quest.« less

Digital case-based learning system in school.

PubMed

Gu, Peipei; Guo, Jiayang

2017-01-01

With the continuing growth of multi-media learning resources, it is important to offer methods helping learners to explore and acquire relevant learning information effectively. As services that organize multi-media learning materials together to support programming learning, the digital case-based learning system is needed. In order to create a case-oriented e-learning system, this paper concentrates on the digital case study of multi-media resources and learning processes with an integrated framework. An integration of multi-media resources, testing and learning strategies recommendation as the learning unit is proposed in the digital case-based learning framework. The learning mechanism of learning guidance, multi-media materials learning and testing feedback is supported in our project. An improved personalized genetic algorithm which incorporates preference information and usage degree into the crossover and mutation process is proposed to assemble the personalized test sheet for each learner. A learning strategies recommendation solution is proposed to recommend learning strategies for learners to help them to learn. The experiments are conducted to prove that the proposed approaches are capable of constructing personalized sheets and the effectiveness of the framework.

Digital case-based learning system in school

PubMed Central

Gu, Peipei

2017-01-01

With the continuing growth of multi-media learning resources, it is important to offer methods helping learners to explore and acquire relevant learning information effectively. As services that organize multi-media learning materials together to support programming learning, the digital case-based learning system is needed. In order to create a case-oriented e-learning system, this paper concentrates on the digital case study of multi-media resources and learning processes with an integrated framework. An integration of multi-media resources, testing and learning strategies recommendation as the learning unit is proposed in the digital case-based learning framework. The learning mechanism of learning guidance, multi-media materials learning and testing feedback is supported in our project. An improved personalized genetic algorithm which incorporates preference information and usage degree into the crossover and mutation process is proposed to assemble the personalized test sheet for each learner. A learning strategies recommendation solution is proposed to recommend learning strategies for learners to help them to learn. The experiments are conducted to prove that the proposed approaches are capable of constructing personalized sheets and the effectiveness of the framework. PMID:29107965

Control-value theory: using achievement emotions to improve understanding of motivation, learning, and performance in medical education: AMEE Guide No. 64.

PubMed

Artino, Anthony R; Holmboe, Eric S; Durning, Steven J

2012-01-01

In this AMEE Guide, we consider the emergent theoretical and empirical work on human emotion and how this work can inform the theory, research, and practice of medical education. In the Guide, we define emotion, in general, and achievement emotions, more specifically. We describe one of the leading contemporary theories of achievement emotions, control-value theory (Pekrun 2006), and we distinguish between different types of achievement emotions, their proximal antecedents, and their consequences for motivation, learning, and performance. Next, we review the empirical support for control-value theory from non-medical fields and suggest several important implications for educational practice. In this section, we highlight the importance of designing learning environments that foster a high degree of control and value for students. Finally, we end with a discussion of the need for more research on achievement emotions in medical education, and we propose several key research questions we believe will facilitate our understanding of achievement emotions and their impact on important educational outcomes.

Neural networks for continuous online learning and control.

PubMed

Choy, Min Chee; Srinivasan, Dipti; Cheu, Ruey Long

2006-11-01

This paper proposes a new hybrid neural network (NN) model that employs a multistage online learning process to solve the distributed control problem with an infinite horizon. Various techniques such as reinforcement learning and evolutionary algorithm are used to design the multistage online learning process. For this paper, the infinite horizon distributed control problem is implemented in the form of real-time distributed traffic signal control for intersections in a large-scale traffic network. The hybrid neural network model is used to design each of the local traffic signal controllers at the respective intersections. As the state of the traffic network changes due to random fluctuation of traffic volumes, the NN-based local controllers will need to adapt to the changing dynamics in order to provide effective traffic signal control and to prevent the traffic network from becoming overcongested. Such a problem is especially challenging if the local controllers are used for an infinite horizon problem where online learning has to take place continuously once the controllers are implemented into the traffic network. A comprehensive simulation model of a section of the Central Business District (CBD) of Singapore has been developed using PARAMICS microscopic simulation program. As the complexity of the simulation increases, results show that the hybrid NN model provides significant improvement in traffic conditions when evaluated against an existing traffic signal control algorithm as well as a new, continuously updated simultaneous perturbation stochastic approximation-based neural network (SPSA-NN). Using the hybrid NN model, the total mean delay of each vehicle has been reduced by 78% and the total mean stoppage time of each vehicle has been reduced by 84% compared to the existing traffic signal control algorithm. This shows the efficacy of the hybrid NN model in solving large-scale traffic signal control problem in a distributed manner. Also, it indicates the possibility of using the hybrid NN model for other applications that are similar in nature as the infinite horizon distributed control problem.

Operant conditioning: a minimal components requirement in artificial spiking neurons designed for bio-inspired robot's controller

PubMed Central

Cyr, André; Boukadoum, Mounir; Thériault, Frédéric

2014-01-01

In this paper, we investigate the operant conditioning (OC) learning process within a bio-inspired paradigm, using artificial spiking neural networks (ASNN) to act as robot brain controllers. In biological agents, OC results in behavioral changes learned from the consequences of previous actions, based on progressive prediction adjustment from rewarding or punishing signals. In a neurorobotics context, virtual and physical autonomous robots may benefit from a similar learning skill when facing unknown and unsupervised environments. In this work, we demonstrate that a simple invariant micro-circuit can sustain OC in multiple learning scenarios. The motivation for this new OC implementation model stems from the relatively complex alternatives that have been described in the computational literature and recent advances in neurobiology. Our elementary kernel includes only a few crucial neurons, synaptic links and originally from the integration of habituation and spike-timing dependent plasticity as learning rules. Using several tasks of incremental complexity, our results show that a minimal neural component set is sufficient to realize many OC procedures. Hence, with the proposed OC module, designing learning tasks with an ASNN and a bio-inspired robot context leads to simpler neural architectures for achieving complex behaviors. PMID:25120464

Operant conditioning: a minimal components requirement in artificial spiking neurons designed for bio-inspired robot's controller.

PubMed

Cyr, André; Boukadoum, Mounir; Thériault, Frédéric

2014-01-01

In this paper, we investigate the operant conditioning (OC) learning process within a bio-inspired paradigm, using artificial spiking neural networks (ASNN) to act as robot brain controllers. In biological agents, OC results in behavioral changes learned from the consequences of previous actions, based on progressive prediction adjustment from rewarding or punishing signals. In a neurorobotics context, virtual and physical autonomous robots may benefit from a similar learning skill when facing unknown and unsupervised environments. In this work, we demonstrate that a simple invariant micro-circuit can sustain OC in multiple learning scenarios. The motivation for this new OC implementation model stems from the relatively complex alternatives that have been described in the computational literature and recent advances in neurobiology. Our elementary kernel includes only a few crucial neurons, synaptic links and originally from the integration of habituation and spike-timing dependent plasticity as learning rules. Using several tasks of incremental complexity, our results show that a minimal neural component set is sufficient to realize many OC procedures. Hence, with the proposed OC module, designing learning tasks with an ASNN and a bio-inspired robot context leads to simpler neural architectures for achieving complex behaviors.

Autonomous learning in humanoid robotics through mental imagery.

PubMed

Di Nuovo, Alessandro G; Marocco, Davide; Di Nuovo, Santo; Cangelosi, Angelo

2013-05-01

In this paper we focus on modeling autonomous learning to improve performance of a humanoid robot through a modular artificial neural networks architecture. A model of a neural controller is presented, which allows a humanoid robot iCub to autonomously improve its sensorimotor skills. This is achieved by endowing the neural controller with a secondary neural system that, by exploiting the sensorimotor skills already acquired by the robot, is able to generate additional imaginary examples that can be used by the controller itself to improve the performance through a simulated mental training. Results and analysis presented in the paper provide evidence of the viability of the approach proposed and help to clarify the rational behind the chosen model and its implementation. Copyright © 2012 Elsevier Ltd. All rights reserved.

Diverse strategy-learning styles promote cooperation in evolutionary spatial prisoner's dilemma game

NASA Astrophysics Data System (ADS)

Liu, Run-Ran; Jia, Chun-Xiao; Rong, Zhihai

2015-11-01

Observational learning and practice learning are two important learning styles and play important roles in our information acquisition. In this paper, we study a spacial evolutionary prisoner's dilemma game, where players can choose the observational learning rule or the practice learning rule when updating their strategies. In the proposed model, we use a parameter p controlling the preference of players choosing the observational learning rule, and found that there exists an optimal value of p leading to the highest cooperation level, which indicates that the cooperation can be promoted by these two learning rules collaboratively and one single learning rule is not favor the promotion of cooperation. By analysing the dynamical behavior of the system, we find that the observational learning rule can make the players residing on cooperative clusters more easily realize the bad sequence of mutual defection. However, a too high observational learning probability suppresses the players to form compact cooperative clusters. Our results highlight the importance of a strategy-updating rule, more importantly, the observational learning rule in the evolutionary cooperation.

A Discussion of Possibility of Reinforcement Learning Using Event-Related Potential in BCI

NASA Astrophysics Data System (ADS)

Yamagishi, Yuya; Tsubone, Tadashi; Wada, Yasuhiro

Recently, Brain computer interface (BCI) which is a direct connecting pathway an external device such as a computer or a robot and a human brain have gotten a lot of attention. Since BCI can control the machines as robots by using the brain activity without using the voluntary muscle, the BCI may become a useful communication tool for handicapped persons, for instance, amyotrophic lateral sclerosis patients. However, in order to realize the BCI system which can perform precise tasks on various environments, it is necessary to design the control rules to adapt to the dynamic environments. Reinforcement learning is one approach of the design of the control rule. If this reinforcement leaning can be performed by the brain activity, it leads to the attainment of BCI that has general versatility. In this research, we paid attention to P300 of event-related potential as an alternative signal of the reward of reinforcement learning. We discriminated between the success and the failure trials from P300 of the EEG of the single trial by using the proposed discrimination algorithm based on Support vector machine. The possibility of reinforcement learning was examined from the viewpoint of the number of discriminated trials. It was shown that there was a possibility to be able to learn in most subjects.

Educational games for brain health: revealing their unexplored potential through a neurocognitive approach.

PubMed

Fissler, Patrick; Kolassa, Iris-Tatjana; Schrader, Claudia

2015-01-01

Educational games link the motivational nature of games with learning of knowledge and skills. Here, we go beyond effects on these learning outcomes. We review two lines of evidence which indicate the currently unexplored potential of educational games to promote brain health: First, gaming with specific neurocognitive demands (e.g., executive control), and second, educational learning experiences (e.g., studying foreign languages) improve brain health markers. These markers include cognitive ability, brain function, and brain structure. As educational games allow the combination of specific neurocognitive demands with educational learning experiences, they seem to be optimally suited for promoting brain health. We propose a neurocognitive approach to reveal this unexplored potential of educational games in future research.

Educational games for brain health: revealing their unexplored potential through a neurocognitive approach

PubMed Central

Fissler, Patrick; Kolassa, Iris-Tatjana; Schrader, Claudia

2015-01-01

Educational games link the motivational nature of games with learning of knowledge and skills. Here, we go beyond effects on these learning outcomes. We review two lines of evidence which indicate the currently unexplored potential of educational games to promote brain health: First, gaming with specific neurocognitive demands (e.g., executive control), and second, educational learning experiences (e.g., studying foreign languages) improve brain health markers. These markers include cognitive ability, brain function, and brain structure. As educational games allow the combination of specific neurocognitive demands with educational learning experiences, they seem to be optimally suited for promoting brain health. We propose a neurocognitive approach to reveal this unexplored potential of educational games in future research. PMID:26257697

Distributed adaptive neural network control for a class of heterogeneous nonlinear multi-agent systems subject to actuation failures

NASA Astrophysics Data System (ADS)

Cui, Bing; Zhao, Chunhui; Ma, Tiedong; Feng, Chi

2017-02-01

In this paper, the cooperative adaptive consensus tracking problem for heterogeneous nonlinear multi-agent systems on directed graph is addressed. Each follower is modelled as a general nonlinear system with the unknown and nonidentical nonlinear dynamics, disturbances and actuator failures. Cooperative fault tolerant neural network tracking controllers with online adaptive learning features are proposed to guarantee that all agents synchronise to the trajectory of one leader with bounded adjustable synchronisation errors. With the help of linear quadratic regulator-based optimal design, a graph-dependent Lyapunov proof provides error bounds that depend on the graph topology, one virtual matrix and some design parameters. Of particular interest is that if the control gain is selected appropriately, the proposed control scheme can be implemented in a unified framework no matter whether there are faults or not. Furthermore, the fault detection and isolation are not needed to implement. Finally, a simulation is given to verify the effectiveness of the proposed method.

A novel model of motor learning capable of developing an optimal movement control law online from scratch.

PubMed

Shimansky, Yury P; Kang, Tao; He, Jiping

2004-02-01

A computational model of a learning system (LS) is described that acquires knowledge and skill necessary for optimal control of a multisegmental limb dynamics (controlled object or CO), starting from "knowing" only the dimensionality of the object's state space. It is based on an optimal control problem setup different from that of reinforcement learning. The LS solves the optimal control problem online while practicing the manipulation of CO. The system's functional architecture comprises several adaptive components, each of which incorporates a number of mapping functions approximated based on artificial neural nets. Besides the internal model of the CO's dynamics and adaptive controller that computes the control law, the LS includes a new type of internal model, the minimal cost (IM(mc)) of moving the controlled object between a pair of states. That internal model appears critical for the LS's capacity to develop an optimal movement trajectory. The IM(mc) interacts with the adaptive controller in a cooperative manner. The controller provides an initial approximation of an optimal control action, which is further optimized in real time based on the IM(mc). The IM(mc) in turn provides information for updating the controller. The LS's performance was tested on the task of center-out reaching to eight randomly selected targets with a 2DOF limb model. The LS reached an optimal level of performance in a few tens of trials. It also quickly adapted to movement perturbations produced by two different types of external force field. The results suggest that the proposed design of a self-optimized control system can serve as a basis for the modeling of motor learning that includes the formation and adaptive modification of the plan of a goal-directed movement.

Structural drift: the population dynamics of sequential learning.

PubMed

Crutchfield, James P; Whalen, Sean

2012-01-01

We introduce a theory of sequential causal inference in which learners in a chain estimate a structural model from their upstream "teacher" and then pass samples from the model to their downstream "student". It extends the population dynamics of genetic drift, recasting Kimura's selectively neutral theory as a special case of a generalized drift process using structured populations with memory. We examine the diffusion and fixation properties of several drift processes and propose applications to learning, inference, and evolution. We also demonstrate how the organization of drift process space controls fidelity, facilitates innovations, and leads to information loss in sequential learning with and without memory.

Brain-machine interface control of a manipulator using small-world neural network and shared control strategy.

PubMed

Li, Ting; Hong, Jun; Zhang, Jinhua; Guo, Feng

2014-03-15

The improvement of the resolution of brain signal and the ability to control external device has been the most important goal in BMI research field. This paper describes a non-invasive brain-actuated manipulator experiment, which defined a paradigm for the motion control of a serial manipulator based on motor imagery and shared control. The techniques of component selection, spatial filtering and classification of motor imagery were involved. Small-world neural network (SWNN) was used to classify five brain states. To verify the effectiveness of the proposed classifier, we replace the SWNN classifier by a radial basis function (RBF) networks neural network, a standard multi-layered feed-forward backpropagation network (SMN) and a multi-SVM classifier, with the same features for the classification. The results also indicate that the proposed classifier achieves a 3.83% improvement over the best results of other classifiers. We proposed a shared control method consisting of two control patterns to expand the control of BMI from the software angle. The job of path building for reaching the 'end' point was designated as an assessment task. We recorded all paths contributed by subjects and picked up relevant parameters as evaluation coefficients. With the assistance of two control patterns and series of machine learning algorithms, the proposed BMI originally achieved the motion control of a manipulator in the whole workspace. According to experimental results, we confirmed the feasibility of the proposed BMI method for 3D motion control of a manipulator using EEG during motor imagery. Copyright © 2013 Elsevier B.V. All rights reserved.

Dynamical Motor Control Learned with Deep Deterministic Policy Gradient

PubMed Central

2018-01-01

Conventional models of motor control exploit the spatial representation of the controlled system to generate control commands. Typically, the control command is gained with the feedback state of a specific instant in time, which behaves like an optimal regulator or spatial filter to the feedback state. Yet, recent neuroscience studies found that the motor network may constitute an autonomous dynamical system and the temporal patterns of the control command can be contained in the dynamics of the motor network, that is, the dynamical system hypothesis (DSH). Inspired by these findings, here we propose a computational model that incorporates this neural mechanism, in which the control command could be unfolded from a dynamical controller whose initial state is specified with the task parameters. The model is trained in a trial-and-error manner in the framework of deep deterministic policy gradient (DDPG). The experimental results show that the dynamical controller successfully learns the control policy for arm reaching movements, while the analysis of the internal activities of the dynamical controller provides the computational evidence to the DSH of the neural coding in motor cortices. PMID:29666634

Dynamical Motor Control Learned with Deep Deterministic Policy Gradient.

PubMed

Shi, Haibo; Sun, Yaoru; Li, Jie

2018-01-01

Conventional models of motor control exploit the spatial representation of the controlled system to generate control commands. Typically, the control command is gained with the feedback state of a specific instant in time, which behaves like an optimal regulator or spatial filter to the feedback state. Yet, recent neuroscience studies found that the motor network may constitute an autonomous dynamical system and the temporal patterns of the control command can be contained in the dynamics of the motor network, that is, the dynamical system hypothesis (DSH). Inspired by these findings, here we propose a computational model that incorporates this neural mechanism, in which the control command could be unfolded from a dynamical controller whose initial state is specified with the task parameters. The model is trained in a trial-and-error manner in the framework of deep deterministic policy gradient (DDPG). The experimental results show that the dynamical controller successfully learns the control policy for arm reaching movements, while the analysis of the internal activities of the dynamical controller provides the computational evidence to the DSH of the neural coding in motor cortices.

Unsupervised Deep Learning Applied to Breast Density Segmentation and Mammographic Risk Scoring.

PubMed

Kallenberg, Michiel; Petersen, Kersten; Nielsen, Mads; Ng, Andrew Y; Pengfei Diao; Igel, Christian; Vachon, Celine M; Holland, Katharina; Winkel, Rikke Rass; Karssemeijer, Nico; Lillholm, Martin

2016-05-01

Mammographic risk scoring has commonly been automated by extracting a set of handcrafted features from mammograms, and relating the responses directly or indirectly to breast cancer risk. We present a method that learns a feature hierarchy from unlabeled data. When the learned features are used as the input to a simple classifier, two different tasks can be addressed: i) breast density segmentation, and ii) scoring of mammographic texture. The proposed model learns features at multiple scales. To control the models capacity a novel sparsity regularizer is introduced that incorporates both lifetime and population sparsity. We evaluated our method on three different clinical datasets. Our state-of-the-art results show that the learned breast density scores have a very strong positive relationship with manual ones, and that the learned texture scores are predictive of breast cancer. The model is easy to apply and generalizes to many other segmentation and scoring problems.

Multi-Site Diagnostic Classification of Schizophrenia Using Discriminant Deep Learning with Functional Connectivity MRI.

PubMed

Zeng, Ling-Li; Wang, Huaning; Hu, Panpan; Yang, Bo; Pu, Weidan; Shen, Hui; Chen, Xingui; Liu, Zhening; Yin, Hong; Tan, Qingrong; Wang, Kai; Hu, Dewen

2018-04-01

A lack of a sufficiently large sample at single sites causes poor generalizability in automatic diagnosis classification of heterogeneous psychiatric disorders such as schizophrenia based on brain imaging scans. Advanced deep learning methods may be capable of learning subtle hidden patterns from high dimensional imaging data, overcome potential site-related variation, and achieve reproducible cross-site classification. However, deep learning-based cross-site transfer classification, despite less imaging site-specificity and more generalizability of diagnostic models, has not been investigated in schizophrenia. A large multi-site functional MRI sample (n = 734, including 357 schizophrenic patients from seven imaging resources) was collected, and a deep discriminant autoencoder network, aimed at learning imaging site-shared functional connectivity features, was developed to discriminate schizophrenic individuals from healthy controls. Accuracies of approximately 85·0% and 81·0% were obtained in multi-site pooling classification and leave-site-out transfer classification, respectively. The learned functional connectivity features revealed dysregulation of the cortical-striatal-cerebellar circuit in schizophrenia, and the most discriminating functional connections were primarily located within and across the default, salience, and control networks. The findings imply that dysfunctional integration of the cortical-striatal-cerebellar circuit across the default, salience, and control networks may play an important role in the "disconnectivity" model underlying the pathophysiology of schizophrenia. The proposed discriminant deep learning method may be capable of learning reliable connectome patterns and help in understanding the pathophysiology and achieving accurate prediction of schizophrenia across multiple independent imaging sites. Copyright © 2018 German Center for Neurodegenerative Diseases (DZNE). Published by Elsevier B.V. All rights reserved.

The effects of traditional learning and a learning cycle inquiry learning strategy on students' science achievement and attitudes toward elementary science

NASA Astrophysics Data System (ADS)

Ebrahim, Ali

The purpose of this study is to examine the impact of two instructional methods on students' academic achievement and attitudes toward elementary science in the State of Kuwait: traditional teaching method and the 4-E learning cycle inquiry teaching method. The subjects were 111 students from four intact 4th grade classes. The experiment group (n = 56) received the learning cycle instruction while the control group (n = 55) received a more traditional approach over a four week period. The same female teacher taught the experimental and control groups for boys and a different female teacher taught experimental and control groups for girls. The dependent variables were measured through the use of: (1) a science achievement test to assess student achievement; and (2) an attitude survey to measure students' attitudes toward science. Quantitative data were collected on students' pre- and post-treatment achievement and attitudes measures. The two way MANOVA reveals that: the 4-E learning cycle instructional method produces significantly greater achievement and attitudes among fourth grade science students than the traditional teaching approach F (2, 93) = 19.765, (P = .000), corresponding to Wilks' Lambda = .702 with an effect size of .298 and a power of 1. In light of these findings, it is therefore suggested that students can achieve greater and have higher science attitudes when the 4-E learning cycle is used. In addition, these findings support the notion that effective instruction in teaching science, such as the 4-E learning cycle instruction, should be proposed and implemented in elementary schools.

A blended-learning programme regarding professional ethics in physiotherapy students.

PubMed

Aguilar-Rodríguez, Marta; Marques-Sule, Elena; Serra-Añó, Pilar; Espí-López, Gemma Victoria; Dueñas-Moscardó, Lirios; Pérez-Alenda, Sofía

2018-01-01

In the university context, assessing students' attitude, knowledge and opinions when applying an innovative methodological approach to teach professional ethics becomes fundamental to know if the used approach is enough motivating for students. To assess the effect of a blended-learning model, based on professional ethics and related to clinical practices, on physiotherapy students' attitude, knowledge and opinions towards learning professional ethics. Research design and participants: A simple-blind clinical trial was performed (NLM identifier NCT03241693) (control group, n = 64; experimental group, n = 65). Both groups followed clinical practices for 8 months. Control group performed a public exposition of a clinical case about professional ethics. By contrast, an 8-month blended-learning programme regarding professional ethics was worked out for experimental group. An online syllabus and online activities were elaborated, while face-to-face active participation techniques were performed to discuss ethical issues. Students' attitudes, knowledge and opinions towards learning professional ethics were assessed. Ethical considerations: The study was approved by the University Ethic Committee of Human Research and followed the ethical principles according to the Declaration of Helsinki. After the programme, attitudes and knowledge towards learning professional ethics of experimental group students significantly improved, while no differences were observed in control group. Moreover, opinions reported an adequate extension of themes and temporization, importance of clinical practices and interest of topics. Case study method and role playing were considered as the most helpful techniques. The blended-learning programme proposed, based on professional ethics and related to clinical practices, improves physiotherapy students' attitudes, knowledge and opinions towards learning professional ethics.

Automatic classification of schizophrenia using resting-state functional language network via an adaptive learning algorithm

NASA Astrophysics Data System (ADS)

Zhu, Maohu; Jie, Nanfeng; Jiang, Tianzi

2014-03-01

A reliable and precise classification of schizophrenia is significant for its diagnosis and treatment of schizophrenia. Functional magnetic resonance imaging (fMRI) is a novel tool increasingly used in schizophrenia research. Recent advances in statistical learning theory have led to applying pattern classification algorithms to access the diagnostic value of functional brain networks, discovered from resting state fMRI data. The aim of this study was to propose an adaptive learning algorithm to distinguish schizophrenia patients from normal controls using resting-state functional language network. Furthermore, here the classification of schizophrenia was regarded as a sample selection problem where a sparse subset of samples was chosen from the labeled training set. Using these selected samples, which we call informative vectors, a classifier for the clinic diagnosis of schizophrenia was established. We experimentally demonstrated that the proposed algorithm incorporating resting-state functional language network achieved 83.6% leaveone- out accuracy on resting-state fMRI data of 27 schizophrenia patients and 28 normal controls. In contrast with KNearest- Neighbor (KNN), Support Vector Machine (SVM) and l1-norm, our method yielded better classification performance. Moreover, our results suggested that a dysfunction of resting-state functional language network plays an important role in the clinic diagnosis of schizophrenia.

Fuzzy Logic Based Anomaly Detection for Embedded Network Security Cyber Sensor

DOE Office of Scientific and Technical Information (OSTI.GOV)

Ondrej Linda; Todd Vollmer; Jason Wright

Resiliency and security in critical infrastructure control systems in the modern world of cyber terrorism constitute a relevant concern. Developing a network security system specifically tailored to the requirements of such critical assets is of a primary importance. This paper proposes a novel learning algorithm for anomaly based network security cyber sensor together with its hardware implementation. The presented learning algorithm constructs a fuzzy logic rule based model of normal network behavior. Individual fuzzy rules are extracted directly from the stream of incoming packets using an online clustering algorithm. This learning algorithm was specifically developed to comply with the constrainedmore » computational requirements of low-cost embedded network security cyber sensors. The performance of the system was evaluated on a set of network data recorded from an experimental test-bed mimicking the environment of a critical infrastructure control system.« less

A Novel Extreme Learning Control Framework of Unmanned Surface Vehicles.

PubMed

Wang, Ning; Sun, Jing-Chao; Er, Meng Joo; Liu, Yan-Cheng

2016-05-01

In this paper, an extreme learning control (ELC) framework using the single-hidden-layer feedforward network (SLFN) with random hidden nodes for tracking an unmanned surface vehicle suffering from unknown dynamics and external disturbances is proposed. By combining tracking errors with derivatives, an error surface and transformed states are defined to encapsulate unknown dynamics and disturbances into a lumped vector field of transformed states. The lumped nonlinearity is further identified accurately by an extreme-learning-machine-based SLFN approximator which does not require a priori system knowledge nor tuning input weights. Only output weights of the SLFN need to be updated by adaptive projection-based laws derived from the Lyapunov approach. Moreover, an error compensator is incorporated to suppress approximation residuals, and thereby contributing to the robustness and global asymptotic stability of the closed-loop ELC system. Simulation studies and comprehensive comparisons demonstrate that the ELC framework achieves high accuracy in both tracking and approximation.

Robotic learning from demonstration of therapist's time-varying assistance to a patient in trajectory-following tasks.

PubMed

Najafi, Mohammad; Adams, Kim; Tavakoli, Mahdi

2017-07-01

The number of people with physical disabilities and impaired motion control is increasing. Consequently, there is a growing demand for intelligent assistive robotic systems to cooperate with people with disability and help them carry out different tasks. To this end, our group has pioneered the use of robot learning from demonstration (RLfD) techniques, which eliminate the need for task-specific robot programming, in robotic rehabilitation and assistive technologies settings. First, in the demonstration phase, the therapist (or in general, a helper) provides an intervention (typically assistance) and cooperatively performs a task with a patient several times. The demonstrated motion is modelled by a statistical RLfD algorithm, which will later be used in the robot controllers to reproduce a similar intervention robotically. In this paper, by proposing a Tangential-Normal Varying-Impedance Controller (TNVIC), the robotic manipulator not only follows the therapist's demonstrated motion, but also mimics his/her interaction impedance during the therapeutic/assistive intervention. The feasibility and efficacy of the proposed framework are evaluated by conducting an experiment involving a healthy adult with cerebral palsy symptoms being induced using transcutaneous electrical nerve stimulation.

A Novel Dynamic Spectrum Access Framework Based on Reinforcement Learning for Cognitive Radio Sensor Networks.

PubMed

Lin, Yun; Wang, Chao; Wang, Jiaxing; Dou, Zheng

2016-10-12

Cognitive radio sensor networks are one of the kinds of application where cognitive techniques can be adopted and have many potential applications, challenges and future research trends. According to the research surveys, dynamic spectrum access is an important and necessary technology for future cognitive sensor networks. Traditional methods of dynamic spectrum access are based on spectrum holes and they have some drawbacks, such as low accessibility and high interruptibility, which negatively affect the transmission performance of the sensor networks. To address this problem, in this paper a new initialization mechanism is proposed to establish a communication link and set up a sensor network without adopting spectrum holes to convey control information. Specifically, firstly a transmission channel model for analyzing the maximum accessible capacity for three different polices in a fading environment is discussed. Secondly, a hybrid spectrum access algorithm based on a reinforcement learning model is proposed for the power allocation problem of both the transmission channel and the control channel. Finally, extensive simulations have been conducted and simulation results show that this new algorithm provides a significant improvement in terms of the tradeoff between the control channel reliability and the efficiency of the transmission channel.

A Novel Dynamic Spectrum Access Framework Based on Reinforcement Learning for Cognitive Radio Sensor Networks

PubMed Central

Lin, Yun; Wang, Chao; Wang, Jiaxing; Dou, Zheng

2016-01-01

Cognitive radio sensor networks are one of the kinds of application where cognitive techniques can be adopted and have many potential applications, challenges and future research trends. According to the research surveys, dynamic spectrum access is an important and necessary technology for future cognitive sensor networks. Traditional methods of dynamic spectrum access are based on spectrum holes and they have some drawbacks, such as low accessibility and high interruptibility, which negatively affect the transmission performance of the sensor networks. To address this problem, in this paper a new initialization mechanism is proposed to establish a communication link and set up a sensor network without adopting spectrum holes to convey control information. Specifically, firstly a transmission channel model for analyzing the maximum accessible capacity for three different polices in a fading environment is discussed. Secondly, a hybrid spectrum access algorithm based on a reinforcement learning model is proposed for the power allocation problem of both the transmission channel and the control channel. Finally, extensive simulations have been conducted and simulation results show that this new algorithm provides a significant improvement in terms of the tradeoff between the control channel reliability and the efficiency of the transmission channel. PMID:27754316

Machine learning for real time remote detection

NASA Astrophysics Data System (ADS)

Labbé, Benjamin; Fournier, Jérôme; Henaff, Gilles; Bascle, Bénédicte; Canu, Stéphane

2010-10-01

Infrared systems are key to providing enhanced capability to military forces such as automatic control of threats and prevention from air, naval and ground attacks. Key requirements for such a system to produce operational benefits are real-time processing as well as high efficiency in terms of detection and false alarm rate. These are serious issues since the system must deal with a large number of objects and categories to be recognized (small vehicles, armored vehicles, planes, buildings, etc.). Statistical learning based algorithms are promising candidates to meet these requirements when using selected discriminant features and real-time implementation. This paper proposes a new decision architecture benefiting from recent advances in machine learning by using an effective method for level set estimation. While building decision function, the proposed approach performs variable selection based on a discriminative criterion. Moreover, the use of level set makes it possible to manage rejection of unknown or ambiguous objects thus preserving the false alarm rate. Experimental evidences reported on real world infrared images demonstrate the validity of our approach.

Switching Reinforcement Learning for Continuous Action Space

NASA Astrophysics Data System (ADS)

Nagayoshi, Masato; Murao, Hajime; Tamaki, Hisashi

Reinforcement Learning (RL) attracts much attention as a technique of realizing computational intelligence such as adaptive and autonomous decentralized systems. In general, however, it is not easy to put RL into practical use. This difficulty includes a problem of designing a suitable action space of an agent, i.e., satisfying two requirements in trade-off: (i) to keep the characteristics (or structure) of an original search space as much as possible in order to seek strategies that lie close to the optimal, and (ii) to reduce the search space as much as possible in order to expedite the learning process. In order to design a suitable action space adaptively, we propose switching RL model to mimic a process of an infant's motor development in which gross motor skills develop before fine motor skills. Then, a method for switching controllers is constructed by introducing and referring to the “entropy”. Further, through computational experiments by using robot navigation problems with one and two-dimensional continuous action space, the validity of the proposed method has been confirmed.

Machine learning algorithms for the creation of clinical healthcare enterprise systems

NASA Astrophysics Data System (ADS)

Mandal, Indrajit

2017-10-01

Clinical recommender systems are increasingly becoming popular for improving modern healthcare systems. Enterprise systems are persuasively used for creating effective nurse care plans to provide nurse training, clinical recommendations and clinical quality control. A novel design of a reliable clinical recommender system based on multiple classifier system (MCS) is implemented. A hybrid machine learning (ML) ensemble based on random subspace method and random forest is presented. The performance accuracy and robustness of proposed enterprise architecture are quantitatively estimated to be above 99% and 97%, respectively (above 95% confidence interval). The study then extends to experimental analysis of the clinical recommender system with respect to the noisy data environment. The ranking of items in nurse care plan is demonstrated using machine learning algorithms (MLAs) to overcome the drawback of the traditional association rule method. The promising experimental results are compared against the sate-of-the-art approaches to highlight the advancement in recommendation technology. The proposed recommender system is experimentally validated using five benchmark clinical data to reinforce the research findings.

A serious game for improving the decision making skills and knowledge levels of Turkish football referees according to the laws of the game.

PubMed

Gulec, Ulas; Yilmaz, Murat

2016-01-01

Digital game-based learning environments provide emerging opportunities to overcome learning barriers by combining newly developed technologies and traditional game design. This study proposes a quantitative research approach supported by expert validation interviews to designing a game-based learning framework. The goal is to improve the learning experience and decision-making skills of soccer referees in Turkey. A serious game was developed and tested on a group of referees (N = 54). The assessment results of these referees were compared with two sample t-test and the Wilcoxon signed-ranked test for both the experimental group and the control group. The findings of the current study confirmed that a game-based learning environment has greater merit over the paper-based alternatives.

Discriminatively learning for representing local image features with quadruplet model

NASA Astrophysics Data System (ADS)

Zhang, Da-long; Zhao, Lei; Xu, Duan-qing; Lu, Dong-ming

2017-11-01

Traditional hand-crafted features for representing local image patches are evolving into current data-driven and learning-based image feature, but learning a robust and discriminative descriptor which is capable of controlling various patch-level computer vision tasks is still an open problem. In this work, we propose a novel deep convolutional neural network (CNN) to learn local feature descriptors. We utilize the quadruplets with positive and negative training samples, together with a constraint to restrict the intra-class variance, to learn good discriminative CNN representations. Compared with previous works, our model reduces the overlap in feature space between corresponding and non-corresponding patch pairs, and mitigates margin varying problem caused by commonly used triplet loss. We demonstrate that our method achieves better embedding result than some latest works, like PN-Net and TN-TG, on benchmark dataset.

Investigation of Proprioceptor Stimulation.

ERIC Educational Resources Information Center

Caukins, Sivan E.; And Others

A research proposal to study the effect of multisensory teaching methods in first-grade reading is presented. The focus is on sex differences in learning and in multisensory approaches to teaching. The project will involve 10 experimental and 10 control first-grade classes in several Southern California schools. Both groups will be given IQ,…

Developing the Concept of Metalearning

ERIC Educational Resources Information Center

Jackson, Norman

2004-01-01

The idea of metalearning was originally used by John Biggs (1985) to describe the state of 'being aware of and taking control of one's own learning'. This paper explores the concept through collaborative enquiry involving researchers and higher education teachers. An evolved conception is proposed in which metalearning is a subconcept within…

Learning of Sensory Sequences in Cerebellar Patients

ERIC Educational Resources Information Center

Frings, Markus; Boenisch, Raoul; Gerwig, Marcus; Diener, Hans-Christoph; Timmann, Dagmar

2004-01-01

A possible role of the cerebellum in detecting and recognizing event sequences has been proposed. The present study sought to determine whether patients with cerebellar lesions are impaired in the acquisition and discrimination of sequences of sensory stimuli of different modalities. A group of 26 cerebellar patients and 26 controls matched for…

The relationship between strategic control and conscious structural knowledge in artificial grammar learning.

PubMed

Norman, Elisabeth; Scott, Ryan B; Price, Mark C; Dienes, Zoltan

2016-05-01

We address Jacoby's (1991) proposal that strategic control over knowledge requires conscious awareness of that knowledge. In a two-grammar artificial grammar learning experiment all participants were trained on two grammars, consisting of a regularity in letter sequences, while two other dimensions (colours and fonts) varied randomly. Strategic control was measured as the ability to selectively apply the grammars during classification. For each classification, participants also made a combined judgement of (a) decision strategy and (b) relevant stimulus dimension. Strategic control was found for all types of decision strategy, including trials where participants claimed to lack conscious structural knowledge. However, strong evidence of strategic control only occurred when participants knew or guessed that the letter dimension was relevant, suggesting that strategic control might be associated with - or even causally requires - global awareness of the nature of the rules even though it does not require detailed knowledge of their content. Copyright © 2016 The Authors. Published by Elsevier Inc. All rights reserved.

Adaptive Critic-based Neurofuzzy Controller for the Steam Generator Water Level

NASA Astrophysics Data System (ADS)

Fakhrazari, Amin; Boroushaki, Mehrdad

2008-06-01

In this paper, an adaptive critic-based neurofuzzy controller is presented for water level regulation of nuclear steam generators. The problem has been of great concern for many years as the steam generator is a highly nonlinear system showing inverse response dynamics especially at low operating power levels. Fuzzy critic-based learning is a reinforcement learning method based on dynamic programming. The only information available for the critic agent is the system feedback which is interpreted as the last action the controller has performed in the previous state. The signal produced by the critic agent is used alongside the backpropagation of error algorithm to tune online conclusion parts of the fuzzy inference rules. The critic agent here has a proportional-derivative structure and the fuzzy rule base has nine rules. The proposed controller shows satisfactory transient responses, disturbance rejection and robustness to model uncertainty. Its simple design procedure and structure, nominates it as one of the suitable controller designs for the steam generator water level control in nuclear power plant industry.

Multilayer Extreme Learning Machine With Subnetwork Nodes for Representation Learning.

PubMed

Yang, Yimin; Wu, Q M Jonathan

2016-11-01

The extreme learning machine (ELM), which was originally proposed for "generalized" single-hidden layer feedforward neural networks, provides efficient unified learning solutions for the applications of clustering, regression, and classification. It presents competitive accuracy with superb efficiency in many applications. However, ELM with subnetwork nodes architecture has not attracted much research attentions. Recently, many methods have been proposed for supervised/unsupervised dimension reduction or representation learning, but these methods normally only work for one type of problem. This paper studies the general architecture of multilayer ELM (ML-ELM) with subnetwork nodes, showing that: 1) the proposed method provides a representation learning platform with unsupervised/supervised and compressed/sparse representation learning and 2) experimental results on ten image datasets and 16 classification datasets show that, compared to other conventional feature learning methods, the proposed ML-ELM with subnetwork nodes performs competitively or much better than other feature learning methods.

Substantia nigra activity level predicts trial-to-trial adjustments in cognitive control

PubMed Central

Boehler, C.N.; Bunzeck, N.; Krebs, R.M.; Noesselt, T.; Schoenfeld, M.A.; Heinze, H.-J.; Münte, T.F.; Woldorff, M.G.; Hopf, J.-M.

2011-01-01

Effective adaptation to the demands of a changing environment requires flexible cognitive control. The medial and lateral frontal cortices are involved in such control processes, putatively in close interplay with the basal ganglia. In particular, dopaminergic projections from the midbrain (i.e., from the substantia nigra (SN) and the ventral tegmental area (VTA)) have been proposed to play a pivotal role in modulating the activity in these areas for cognitive control purposes. In that dopaminergic involvement has been strongly implicated in reinforcement learning, these ideas suggest functional links between reinforcement learning, where the outcome of actions shapes behavior over time, and cognitive control in a more general context, where no direct reward is involved. Here, we provide evidence from functional MRI in humans that activity in the SN predicts systematic subsequent trial-to-trial response time (RT) prolongations that are thought to reflect cognitive control in a Stop-signal paradigm. In particular, variations in the activity level of the SN in one trial predicted the degree of RT prolongation on the subsequent trial, consistent with a modulating output signal from the SN being involved in enhancing cognitive control. This link between SN activity and subsequent behavioral adjustments lends support to theoretical accounts that propose dopaminergic control signals that shape behavior both in the presence and absence of direct reward. This SN-based modulatory mechanism is presumably mediated via a wider network that determines response speed in this task, including frontal and parietal control regions, along with the basal ganglia and the associated subthalamic nucleus. PMID:20465358

Intrinsic Sensing and Evolving Internal Model Control of Compact Elastic Module for a Lower Extremity Exoskeleton

PubMed Central

Wang, Likun; Du, Zhijiang; Dong, Wei; Shen, Yi; Zhao, Guangyu

2018-01-01

To achieve strength augmentation, endurance enhancement, and human assistance in a functional autonomous exoskeleton, control precision, back drivability, low output impedance, and mechanical compactness are desired. In our previous work, two elastic modules were designed for human–robot interaction sensing and compliant control, respectively. According to the intrinsic sensing properties of the elastic module, in this paper, only one compact elastic module is applied to realize both purposes. Thus, the corresponding control strategy is required and evolving internal model control is proposed to address this issue. Moreover, the input signal to the controller is derived from the deflection of the compact elastic module. The human–robot interaction is considered as the disturbance which is approximated by the output error between the exoskeleton control plant and evolving forward learning model. Finally, to verify our proposed control scheme, several experiments are conducted with our robotic exoskeleton system. The experiment shows a satisfying result and promising application feasibility. PMID:29562684

Intrinsic Sensing and Evolving Internal Model Control of Compact Elastic Module for a Lower Extremity Exoskeleton.

PubMed

Wang, Likun; Du, Zhijiang; Dong, Wei; Shen, Yi; Zhao, Guangyu

2018-03-19

To achieve strength augmentation, endurance enhancement, and human assistance in a functional autonomous exoskeleton, control precision, back drivability, low output impedance, and mechanical compactness are desired. In our previous work, two elastic modules were designed for human-robot interaction sensing and compliant control, respectively. According to the intrinsic sensing properties of the elastic module, in this paper, only one compact elastic module is applied to realize both purposes. Thus, the corresponding control strategy is required and evolving internal model control is proposed to address this issue. Moreover, the input signal to the controller is derived from the deflection of the compact elastic module. The human-robot interaction is considered as the disturbance which is approximated by the output error between the exoskeleton control plant and evolving forward learning model. Finally, to verify our proposed control scheme, several experiments are conducted with our robotic exoskeleton system. The experiment shows a satisfying result and promising application feasibility.

Approximate kernel competitive learning.

PubMed

Wu, Jian-Sheng; Zheng, Wei-Shi; Lai, Jian-Huang

2015-03-01

Kernel competitive learning has been successfully used to achieve robust clustering. However, kernel competitive learning (KCL) is not scalable for large scale data processing, because (1) it has to calculate and store the full kernel matrix that is too large to be calculated and kept in the memory and (2) it cannot be computed in parallel. In this paper we develop a framework of approximate kernel competitive learning for processing large scale dataset. The proposed framework consists of two parts. First, it derives an approximate kernel competitive learning (AKCL), which learns kernel competitive learning in a subspace via sampling. We provide solid theoretical analysis on why the proposed approximation modelling would work for kernel competitive learning, and furthermore, we show that the computational complexity of AKCL is largely reduced. Second, we propose a pseudo-parallelled approximate kernel competitive learning (PAKCL) based on a set-based kernel competitive learning strategy, which overcomes the obstacle of using parallel programming in kernel competitive learning and significantly accelerates the approximate kernel competitive learning for large scale clustering. The empirical evaluation on publicly available datasets shows that the proposed AKCL and PAKCL can perform comparably as KCL, with a large reduction on computational cost. Also, the proposed methods achieve more effective clustering performance in terms of clustering precision against related approximate clustering approaches. Copyright © 2014 Elsevier Ltd. All rights reserved.

Optimal and Scalable Caching for 5G Using Reinforcement Learning of Space-Time Popularities

NASA Astrophysics Data System (ADS)

Sadeghi, Alireza; Sheikholeslami, Fatemeh; Giannakis, Georgios B.

2018-02-01

Small basestations (SBs) equipped with caching units have potential to handle the unprecedented demand growth in heterogeneous networks. Through low-rate, backhaul connections with the backbone, SBs can prefetch popular files during off-peak traffic hours, and service them to the edge at peak periods. To intelligently prefetch, each SB must learn what and when to cache, while taking into account SB memory limitations, the massive number of available contents, the unknown popularity profiles, as well as the space-time popularity dynamics of user file requests. In this work, local and global Markov processes model user requests, and a reinforcement learning (RL) framework is put forth for finding the optimal caching policy when the transition probabilities involved are unknown. Joint consideration of global and local popularity demands along with cache-refreshing costs allow for a simple, yet practical asynchronous caching approach. The novel RL-based caching relies on a Q-learning algorithm to implement the optimal policy in an online fashion, thus enabling the cache control unit at the SB to learn, track, and possibly adapt to the underlying dynamics. To endow the algorithm with scalability, a linear function approximation of the proposed Q-learning scheme is introduced, offering faster convergence as well as reduced complexity and memory requirements. Numerical tests corroborate the merits of the proposed approach in various realistic settings.

Instructional control of reinforcement learning: A behavioral and neurocomputational investigation

PubMed Central

Doll, Bradley B.; Jacobs, W. Jake; Sanfey, Alan G.; Frank, Michael J.

2011-01-01

Humans learn how to behave directly through environmental experience and indirectly through rules and instructions. Behavior analytic research has shown that instructions can control behavior, even when such behavior leads to sub-optimal outcomes (Hayes, S. (Ed.). 1989. Rule-governed behavior: cognition, contingencies, and instructional control. Plenum Press.). Here we examine the control of behavior through instructions in a reinforcement learning task known to depend on striatal dopaminergic function. Participants selected between probabilistically reinforced stimuli, and were (incorrectly) told that a specific stimulus had the highest (or lowest) reinforcement probability. Despite experience to the contrary, instructions drove choice behavior. We present neural network simulations that capture the interactions between instruction-driven and reinforcement-driven behavior via two potential neural circuits: one in which the striatum is inaccurately trained by instruction representations coming from prefrontal cortex/hippocampus (PFC/HC), and another in which the striatum learns the environmentally based reinforcement contingencies, but is “overridden” at decision output. Both models capture the core behavioral phenomena but, because they differ fundamentally on what is learned, make distinct predictions for subsequent behavioral and neuroimaging experiments. Finally, we attempt to distinguish between the proposed computational mechanisms governing instructed behavior by fitting a series of abstract “Q-learning” and Bayesian models to subject data. The best-fitting model supports one of the neural models, suggesting the existence of a “confirmation bias” in which the PFC/HC system trains the reinforcement system by amplifying outcomes that are consistent with instructions while diminishing inconsistent outcomes. PMID:19595993

A machine learning-based framework to identify type 2 diabetes through electronic health records

PubMed Central

Zheng, Tao; Xie, Wei; Xu, Liling; He, Xiaoying; Zhang, Ya; You, Mingrong; Yang, Gong; Chen, You

2016-01-01

Objective To discover diverse genotype-phenotype associations affiliated with Type 2 Diabetes Mellitus (T2DM) via genome-wide association study (GWAS) and phenome-wide association study (PheWAS), more cases (T2DM subjects) and controls (subjects without T2DM) are required to be identified (e.g., via Electronic Health Records (EHR)). However, existing expert based identification algorithms often suffer in a low recall rate and could miss a large number of valuable samples under conservative filtering standards. The goal of this work is to develop a semi-automated framework based on machine learning as a pilot study to liberalize filtering criteria to improve recall rate with a keeping of low false positive rate. Materials and methods We propose a data informed framework for identifying subjects with and without T2DM from EHR via feature engineering and machine learning. We evaluate and contrast the identification performance of widely-used machine learning models within our framework, including k-Nearest-Neighbors, Naïve Bayes, Decision Tree, Random Forest, Support Vector Machine and Logistic Regression. Our framework was conducted on 300 patient samples (161 cases, 60 controls and 79 unconfirmed subjects), randomly selected from 23,281 diabetes related cohort retrieved from a regional distributed EHR repository ranging from 2012 to 2014. Results We apply top-performing machine learning algorithms on the engineered features. We benchmark and contrast the accuracy, precision, AUC, sensitivity and specificity of classification models against the state-of-the-art expert algorithm for identification of T2DM subjects. Our results indicate that the framework achieved high identification performances (∼0.98 in average AUC), which are much higher than the state-of-the-art algorithm (0.71 in AUC). Discussion Expert algorithm-based identification of T2DM subjects from EHR is often hampered by the high missing rates due to their conservative selection criteria. Our framework leverages machine learning and feature engineering to loosen such selection criteria to achieve a high identification rate of cases and controls. Conclusions Our proposed framework demonstrates a more accurate and efficient approach for identifying subjects with and without T2DM from EHR. PMID:27919371

A machine learning-based framework to identify type 2 diabetes through electronic health records.

PubMed

Zheng, Tao; Xie, Wei; Xu, Liling; He, Xiaoying; Zhang, Ya; You, Mingrong; Yang, Gong; Chen, You

2017-01-01

To discover diverse genotype-phenotype associations affiliated with Type 2 Diabetes Mellitus (T2DM) via genome-wide association study (GWAS) and phenome-wide association study (PheWAS), more cases (T2DM subjects) and controls (subjects without T2DM) are required to be identified (e.g., via Electronic Health Records (EHR)). However, existing expert based identification algorithms often suffer in a low recall rate and could miss a large number of valuable samples under conservative filtering standards. The goal of this work is to develop a semi-automated framework based on machine learning as a pilot study to liberalize filtering criteria to improve recall rate with a keeping of low false positive rate. We propose a data informed framework for identifying subjects with and without T2DM from EHR via feature engineering and machine learning. We evaluate and contrast the identification performance of widely-used machine learning models within our framework, including k-Nearest-Neighbors, Naïve Bayes, Decision Tree, Random Forest, Support Vector Machine and Logistic Regression. Our framework was conducted on 300 patient samples (161 cases, 60 controls and 79 unconfirmed subjects), randomly selected from 23,281 diabetes related cohort retrieved from a regional distributed EHR repository ranging from 2012 to 2014. We apply top-performing machine learning algorithms on the engineered features. We benchmark and contrast the accuracy, precision, AUC, sensitivity and specificity of classification models against the state-of-the-art expert algorithm for identification of T2DM subjects. Our results indicate that the framework achieved high identification performances (∼0.98 in average AUC), which are much higher than the state-of-the-art algorithm (0.71 in AUC). Expert algorithm-based identification of T2DM subjects from EHR is often hampered by the high missing rates due to their conservative selection criteria. Our framework leverages machine learning and feature engineering to loosen such selection criteria to achieve a high identification rate of cases and controls. Our proposed framework demonstrates a more accurate and efficient approach for identifying subjects with and without T2DM from EHR. Copyright © 2016 Elsevier Ireland Ltd. All rights reserved.

Multi-Source Learning for Joint Analysis of Incomplete Multi-Modality Neuroimaging Data

PubMed Central

Yuan, Lei; Wang, Yalin; Thompson, Paul M.; Narayan, Vaibhav A.; Ye, Jieping

2013-01-01

Incomplete data present serious problems when integrating largescale brain imaging data sets from different imaging modalities. In the Alzheimer’s Disease Neuroimaging Initiative (ADNI), for example, over half of the subjects lack cerebrospinal fluid (CSF) measurements; an independent half of the subjects do not have fluorodeoxyglucose positron emission tomography (FDG-PET) scans; many lack proteomics measurements. Traditionally, subjects with missing measures are discarded, resulting in a severe loss of available information. We address this problem by proposing two novel learning methods where all the samples (with at least one available data source) can be used. In the first method, we divide our samples according to the availability of data sources, and we learn shared sets of features with state-of-the-art sparse learning methods. Our second method learns a base classifier for each data source independently, based on which we represent each source using a single column of prediction scores; we then estimate the missing prediction scores, which, combined with the existing prediction scores, are used to build a multi-source fusion model. To illustrate the proposed approaches, we classify patients from the ADNI study into groups with Alzheimer’s disease (AD), mild cognitive impairment (MCI) and normal controls, based on the multi-modality data. At baseline, ADNI’s 780 participants (172 AD, 397 MCI, 211 Normal), have at least one of four data types: magnetic resonance imaging (MRI), FDG-PET, CSF and proteomics. These data are used to test our algorithms. Comprehensive experiments show that our proposed methods yield stable and promising results. PMID:24014189

A Circuit-Based Neural Network with Hybrid Learning of Backpropagation and Random Weight Change Algorithms

PubMed Central

Yang, Changju; Kim, Hyongsuk; Adhikari, Shyam Prasad; Chua, Leon O.

2016-01-01

A hybrid learning method of a software-based backpropagation learning and a hardware-based RWC learning is proposed for the development of circuit-based neural networks. The backpropagation is known as one of the most efficient learning algorithms. A weak point is that its hardware implementation is extremely difficult. The RWC algorithm, which is very easy to implement with respect to its hardware circuits, takes too many iterations for learning. The proposed learning algorithm is a hybrid one of these two. The main learning is performed with a software version of the BP algorithm, firstly, and then, learned weights are transplanted on a hardware version of a neural circuit. At the time of the weight transplantation, a significant amount of output error would occur due to the characteristic difference between the software and the hardware. In the proposed method, such error is reduced via a complementary learning of the RWC algorithm, which is implemented in a simple hardware. The usefulness of the proposed hybrid learning system is verified via simulations upon several classical learning problems. PMID:28025566

Attentional load and implicit sequence learning.

PubMed

Shanks, David R; Rowland, Lee A; Ranger, Mandeep S

2005-06-01

A widely employed conceptualization of implicit learning hypothesizes that it makes minimal demands on attentional resources. This conjecture was investigated by comparing learning under single-task and dual-task conditions in the sequential reaction time (SRT) task. Participants learned probabilistic sequences, with dual-task participants additionally having to perform a counting task using stimuli that were targets in the SRT display. Both groups were then tested for sequence knowledge under single-task (Experiments 1 and 2) or dual-task (Experiment 3) conditions. Participants also completed a free generation task (Experiments 2 and 3) under inclusion or exclusion conditions to determine if sequence knowledge was conscious or unconscious in terms of its access to intentional control. The experiments revealed that the secondary task impaired sequence learning and that sequence knowledge was consciously accessible. These findings disconfirm both the notion that implicit learning is able to proceed normally under conditions of divided attention, and that the acquired knowledge is inaccessible to consciousness. A unitary framework for conceptualizing implicit and explicit learning is proposed.

Exploration of joint redundancy but not task space variability facilitates supervised motor learning.

PubMed

Singh, Puneet; Jana, Sumitash; Ghosal, Ashitava; Murthy, Aditya

2016-12-13

The number of joints and muscles in a human arm is more than what is required for reaching to a desired point in 3D space. Although previous studies have emphasized how such redundancy and the associated flexibility may play an important role in path planning, control of noise, and optimization of motion, whether and how redundancy might promote motor learning has not been investigated. In this work, we quantify redundancy space and investigate its significance and effect on motor learning. We propose that a larger redundancy space leads to faster learning across subjects. We observed this pattern in subjects learning novel kinematics (visuomotor adaptation) and dynamics (force-field adaptation). Interestingly, we also observed differences in the redundancy space between the dominant hand and nondominant hand that explained differences in the learning of dynamics. Taken together, these results provide support for the hypothesis that redundancy aids in motor learning and that the redundant component of motor variability is not noise.

Exploration of joint redundancy but not task space variability facilitates supervised motor learning

PubMed Central

Singh, Puneet; Jana, Sumitash; Ghosal, Ashitava; Murthy, Aditya

2016-01-01

The number of joints and muscles in a human arm is more than what is required for reaching to a desired point in 3D space. Although previous studies have emphasized how such redundancy and the associated flexibility may play an important role in path planning, control of noise, and optimization of motion, whether and how redundancy might promote motor learning has not been investigated. In this work, we quantify redundancy space and investigate its significance and effect on motor learning. We propose that a larger redundancy space leads to faster learning across subjects. We observed this pattern in subjects learning novel kinematics (visuomotor adaptation) and dynamics (force-field adaptation). Interestingly, we also observed differences in the redundancy space between the dominant hand and nondominant hand that explained differences in the learning of dynamics. Taken together, these results provide support for the hypothesis that redundancy aids in motor learning and that the redundant component of motor variability is not noise. PMID:27911808

Partitioned learning of deep Boltzmann machines for SNP data.

PubMed

Hess, Moritz; Lenz, Stefan; Blätte, Tamara J; Bullinger, Lars; Binder, Harald

2017-10-15

Learning the joint distributions of measurements, and in particular identification of an appropriate low-dimensional manifold, has been found to be a powerful ingredient of deep leaning approaches. Yet, such approaches have hardly been applied to single nucleotide polymorphism (SNP) data, probably due to the high number of features typically exceeding the number of studied individuals. After a brief overview of how deep Boltzmann machines (DBMs), a deep learning approach, can be adapted to SNP data in principle, we specifically present a way to alleviate the dimensionality problem by partitioned learning. We propose a sparse regression approach to coarsely screen the joint distribution of SNPs, followed by training several DBMs on SNP partitions that were identified by the screening. Aggregate features representing SNP patterns and the corresponding SNPs are extracted from the DBMs by a combination of statistical tests and sparse regression. In simulated case-control data, we show how this can uncover complex SNP patterns and augment results from univariate approaches, while maintaining type 1 error control. Time-to-event endpoints are considered in an application with acute myeloid leukemia patients, where SNP patterns are modeled after a pre-screening based on gene expression data. The proposed approach identified three SNPs that seem to jointly influence survival in a validation dataset. This indicates the added value of jointly investigating SNPs compared to standard univariate analyses and makes partitioned learning of DBMs an interesting complementary approach when analyzing SNP data. A Julia package is provided at 'http://github.com/binderh/BoltzmannMachines.jl'. binderh@imbi.uni-freiburg.de. Supplementary data are available at Bioinformatics online. © The Author (2017). Published by Oxford University Press. All rights reserved. For Permissions, please email: journals.permissions@oup.com

Use of Machine Learning Classifiers and Sensor Data to Detect Neurological Deficit in Stroke Patients.

PubMed

Park, Eunjeong; Chang, Hyuk-Jae; Nam, Hyo Suk

2017-04-18

The pronator drift test (PDT), a neurological examination, is widely used in clinics to measure motor weakness of stroke patients. The aim of this study was to develop a PDT tool with machine learning classifiers to detect stroke symptoms based on quantification of proximal arm weakness using inertial sensors and signal processing. We extracted features of drift and pronation from accelerometer signals of wearable devices on the inner wrists of 16 stroke patients and 10 healthy controls. Signal processing and feature selection approach were applied to discriminate PDT features used to classify stroke patients. A series of machine learning techniques, namely support vector machine (SVM), radial basis function network (RBFN), and random forest (RF), were implemented to discriminate stroke patients from controls with leave-one-out cross-validation. Signal processing by the PDT tool extracted a total of 12 PDT features from sensors. Feature selection abstracted the major attributes from the 12 PDT features to elucidate the dominant characteristics of proximal weakness of stroke patients using machine learning classification. Our proposed PDT classifiers had an area under the receiver operating characteristic curve (AUC) of .806 (SVM), .769 (RBFN), and .900 (RF) without feature selection, and feature selection improves the AUCs to .913 (SVM), .956 (RBFN), and .975 (RF), representing an average performance enhancement of 15.3%. Sensors and machine learning methods can reliably detect stroke signs and quantify proximal arm weakness. Our proposed solution will facilitate pervasive monitoring of stroke patients. ©Eunjeong Park, Hyuk-Jae Chang, Hyo Suk Nam. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 18.04.2017.

Adaptive learning in a compartmental model of visual cortex—how feedback enables stable category learning and refinement

PubMed Central

Layher, Georg; Schrodt, Fabian; Butz, Martin V.; Neumann, Heiko

2014-01-01

The categorization of real world objects is often reflected in the similarity of their visual appearances. Such categories of objects do not necessarily form disjunct sets of objects, neither semantically nor visually. The relationship between categories can often be described in terms of a hierarchical structure. For instance, tigers and leopards build two separate mammalian categories, both of which are subcategories of the category Felidae. In the last decades, the unsupervised learning of categories of visual input stimuli has been addressed by numerous approaches in machine learning as well as in computational neuroscience. However, the question of what kind of mechanisms might be involved in the process of subcategory learning, or category refinement, remains a topic of active investigation. We propose a recurrent computational network architecture for the unsupervised learning of categorial and subcategorial visual input representations. During learning, the connection strengths of bottom-up weights from input to higher-level category representations are adapted according to the input activity distribution. In a similar manner, top-down weights learn to encode the characteristics of a specific stimulus category. Feedforward and feedback learning in combination realize an associative memory mechanism, enabling the selective top-down propagation of a category's feedback weight distribution. We suggest that the difference between the expected input encoded in the projective field of a category node and the current input pattern controls the amplification of feedforward-driven representations. Large enough differences trigger the recruitment of new representational resources and the establishment of additional (sub-) category representations. We demonstrate the temporal evolution of such learning and show how the proposed combination of an associative memory with a modulatory feedback integration successfully establishes category and subcategory representations. PMID:25538637

Nonuniformity correction for an infrared focal plane array based on diamond search block matching.

PubMed

Sheng-Hui, Rong; Hui-Xin, Zhou; Han-Lin, Qin; Rui, Lai; Kun, Qian

2016-05-01

In scene-based nonuniformity correction algorithms, artificial ghosting and image blurring degrade the correction quality severely. In this paper, an improved algorithm based on the diamond search block matching algorithm and the adaptive learning rate is proposed. First, accurate transform pairs between two adjacent frames are estimated by the diamond search block matching algorithm. Then, based on the error between the corresponding transform pairs, the gradient descent algorithm is applied to update correction parameters. During the process of gradient descent, the local standard deviation and a threshold are utilized to control the learning rate to avoid the accumulation of matching error. Finally, the nonuniformity correction would be realized by a linear model with updated correction parameters. The performance of the proposed algorithm is thoroughly studied with four real infrared image sequences. Experimental results indicate that the proposed algorithm can reduce the nonuniformity with less ghosting artifacts in moving areas and can also overcome the problem of image blurring in static areas.

A Proposed Methodology for the Control of a Semi-Robotic Convoy

DTIC Science & Technology

1991-01-01

verifies that the convoy is controlled within the specifications of the system. 0 Acknowledgements There are numerous people to whom I owe a great deal of...insurmountable problems almost * trivial. Thanks. It would not be complete without thanking the people who kept the world in per- iii 0 spective. Steve... succesive position samples of the lead vehicle, the velocity information is obtained. With this information, the trailing vehicles can repeat the learned

Classifying BCI signals from novice users with extreme learning machine

NASA Astrophysics Data System (ADS)

Rodríguez-Bermúdez, Germán; Bueno-Crespo, Andrés; José Martinez-Albaladejo, F.

2017-07-01

Brain computer interface (BCI) allows to control external devices only with the electrical activity of the brain. In order to improve the system, several approaches have been proposed. However it is usual to test algorithms with standard BCI signals from experts users or from repositories available on Internet. In this work, extreme learning machine (ELM) has been tested with signals from 5 novel users to compare with standard classification algorithms. Experimental results show that ELM is a suitable method to classify electroencephalogram signals from novice users.

Cognitive control predicts use of model-based reinforcement learning.

PubMed

Otto, A Ross; Skatova, Anya; Madlon-Kay, Seth; Daw, Nathaniel D

2015-02-01

Accounts of decision-making and its neural substrates have long posited the operation of separate, competing valuation systems in the control of choice behavior. Recent theoretical and experimental work suggest that this classic distinction between behaviorally and neurally dissociable systems for habitual and goal-directed (or more generally, automatic and controlled) choice may arise from two computational strategies for reinforcement learning (RL), called model-free and model-based RL, but the cognitive or computational processes by which one system may dominate over the other in the control of behavior is a matter of ongoing investigation. To elucidate this question, we leverage the theoretical framework of cognitive control, demonstrating that individual differences in utilization of goal-related contextual information--in the service of overcoming habitual, stimulus-driven responses--in established cognitive control paradigms predict model-based behavior in a separate, sequential choice task. The behavioral correspondence between cognitive control and model-based RL compellingly suggests that a common set of processes may underpin the two behaviors. In particular, computational mechanisms originally proposed to underlie controlled behavior may be applicable to understanding the interactions between model-based and model-free choice behavior.

Novel neural control for a class of uncertain pure-feedback systems.

PubMed

Shen, Qikun; Shi, Peng; Zhang, Tianping; Lim, Cheng-Chew

2014-04-01

This paper is concerned with the problem of adaptive neural tracking control for a class of uncertain pure-feedback nonlinear systems. Using the implicit function theorem and backstepping technique, a practical robust adaptive neural control scheme is proposed to guarantee that the tracking error converges to an adjusted neighborhood of the origin by choosing appropriate design parameters. In contrast to conventional Lyapunov-based design techniques, an alternative Lyapunov function is constructed for the development of control law and learning algorithms. Differing from the existing results in the literature, the control scheme does not need to compute the derivatives of virtual control signals at each step in backstepping design procedures. Furthermore, the scheme requires the desired trajectory and its first derivative rather than its first n derivatives. In addition, the useful property of the basis function of the radial basis function, which will be used in control design, is explored. Simulation results illustrate the effectiveness of the proposed techniques.

Improving learning performance with happiness by interactive scenarios.

PubMed

Chuang, Chi-Hung; Chen, Ying-Nong; Tsai, Luo-Wei; Lee, Chun-Chieh; Tsai, Hsin-Chun

2014-01-01

Recently, digital learning has attracted a lot of researchers to improve the problems of learning carelessness, low learning ability, lack of concentration, and difficulties in comprehending the logic of math. In this study, a digital learning system based on Kinect somatosensory system is proposed to make children and teenagers happily learn in the course of the games and improve the learning performance. We propose two interactive geometry and puzzle games. The proposed somatosensory games can make learners feel curious and raise their motivation to find solutions for boring problems via abundant physical expressions and interactive operations. The players are asked to select particular operation by gestures and physical expressions within a certain time. By doing so, the learners can feel the fun of game playing and train their logic ability before they are aware. Experimental results demonstrate that the proposed somatosensory system can effectively improve the students' learning performance.

Improving Learning Performance with Happiness by Interactive Scenarios

PubMed Central

Chuang, Chi-Hung; Chen, Ying-Nong; Tsai, Luo-Wei; Lee, Chun-Chieh; Tsai, Hsin-Chun

2014-01-01

Recently, digital learning has attracted a lot of researchers to improve the problems of learning carelessness, low learning ability, lack of concentration, and difficulties in comprehending the logic of math. In this study, a digital learning system based on Kinect somatosensory system is proposed to make children and teenagers happily learn in the course of the games and improve the learning performance. We propose two interactive geometry and puzzle games. The proposed somatosensory games can make learners feel curious and raise their motivation to find solutions for boring problems via abundant physical expressions and interactive operations. The players are asked to select particular operation by gestures and physical expressions within a certain time. By doing so, the learners can feel the fun of game playing and train their logic ability before they are aware. Experimental results demonstrate that the proposed somatosensory system can effectively improve the students' learning performance. PMID:24558331

Enhanced Muscle Afferent Signals during Motor Learning in Humans.

PubMed

Dimitriou, Michael

2016-04-25

Much has been revealed concerning human motor learning at the behavioral level [1, 2], but less is known about changes in the involved neural circuits and signals. By examining muscle spindle responses during a classic visuomotor adaptation task [3-6] performed by fully alert humans, I found substantial modulation of sensory afferent signals as a function of adaptation state. Specifically, spindle control was independent of concurrent muscle activity but was specific to movement direction (representing muscle lengthening versus shortening) and to different stages of learning. Increased spindle afferent responses to muscle stretch occurring early during learning reflected individual error size and were negatively related to subsequent antagonist activity (i.e., 60-80 ms thereafter). Relative increases in tonic afferent output early during learning were predictive of the subjects' adaptation rate. I also found that independent spindle control during sensory realignment (the "washout" stage) induced afferent signal "linearization" with respect to muscle length (i.e., signals were more tuned to hand position). The results demonstrate for the first time that motor learning also involves independent and state-related modulation of sensory mechanoreceptor signals. The current findings suggest that adaptive motor performance also relies on the independent control of sensors, not just of muscles. I propose that the "γ" motor system innervating spindles acts to facilitate the acquisition and extraction of task-relevant information at the early stages of sensorimotor adaptation. This designates a more active and targeted role for the human proprioceptive system during motor learning. Copyright © 2016 Elsevier Ltd. All rights reserved.

Learning compliant manipulation through kinesthetic and tactile human-robot interaction.

PubMed

Kronander, Klas; Billard, Aude

2014-01-01

Robot Learning from Demonstration (RLfD) has been identified as a key element for making robots useful in daily lives. A wide range of techniques has been proposed for deriving a task model from a set of demonstrations of the task. Most previous works use learning to model the kinematics of the task, and for autonomous execution the robot then relies on a stiff position controller. While many tasks can and have been learned this way, there are tasks in which controlling the position alone is insufficient to achieve the goals of the task. These are typically tasks that involve contact or require a specific response to physical perturbations. The question of how to adjust the compliance to suit the need of the task has not yet been fully treated in Robot Learning from Demonstration. In this paper, we address this issue and present interfaces that allow a human teacher to indicate compliance variations by physically interacting with the robot during task execution. We validate our approach in two different experiments on the 7 DoF Barrett WAM and KUKA LWR robot manipulators. Furthermore, we conduct a user study to evaluate the usability of our approach from a non-roboticists perspective.

A proof-of-principle simulation for closed-loop control based on preexisting experimental thalamic DBS-enhanced instrumental learning.

PubMed

Wang, Ching-Fu; Yang, Shih-Hung; Lin, Sheng-Huang; Chen, Po-Chuan; Lo, Yu-Chun; Pan, Han-Chi; Lai, Hsin-Yi; Liao, Lun-De; Lin, Hui-Ching; Chen, Hsu-Yan; Huang, Wei-Chen; Huang, Wun-Jhu; Chen, You-Yin

Deep brain stimulation (DBS) has been applied as an effective therapy for treating Parkinson's disease or essential tremor. Several open-loop DBS control strategies have been developed for clinical experiments, but they are limited by short battery life and inefficient therapy. Therefore, many closed-loop DBS control systems have been designed to tackle these problems by automatically adjusting the stimulation parameters via feedback from neural signals, which has been reported to reduce the power consumption. However, when the association between the biomarkers of the model and stimulation is unclear, it is difficult to develop an optimal control scheme for other DBS applications, i.e., DBS-enhanced instrumental learning. Furthermore, few studies have investigated the effect of closed-loop DBS control for cognition function, such as instrumental skill learning, and have been implemented in simulation environments. In this paper, we proposed a proof-of-principle design for a closed-loop DBS system, cognitive-enhancing DBS (ceDBS), which enhanced skill learning based on in vivo experimental data. The ceDBS acquired local field potential (LFP) signal from the thalamic central lateral (CL) nuclei of animals through a neural signal processing system. A strong coupling of the theta oscillation (4-7 Hz) and the learning period was found in the water reward-related lever-pressing learning task. Therefore, the theta-band power ratio, which was the averaged theta band to averaged total band (1-55 Hz) power ratio, could be used as a physiological marker for enhancement of instrumental skill learning. The on-line extraction of the theta-band power ratio was implemented on a field-programmable gate array (FPGA). An autoregressive with exogenous inputs (ARX)-based predictor was designed to construct a CL-thalamic DBS model and forecast the future physiological marker according to the past physiological marker and applied DBS. The prediction could further assist the design of a closed-loop DBS controller. A DBS controller based on a fuzzy expert system was devised to automatically control DBS according to the predicted physiological marker via a set of rules. The simulated experimental results demonstrate that the ceDBS based on the closed-loop control architecture not only reduced power consumption using the predictive physiological marker, but also achieved a desired level of physiological marker through the DBS controller. Copyright © 2017 Elsevier Inc. All rights reserved.

Handwriting assessment to distinguish comorbid learning difficulties from attention deficit hyperactivity disorder in Chinese adolescents: A case-control study.

PubMed

Li-Tsang, Cecilia W P; Li, Tim M H; Lau, Mandy S W; Ho, Choco H Y; Leung, Howard W H

2018-05-15

Attention deficit hyperactivity disorder (ADHD) and learning difficulties (LDs) are proposed as 2 overlapping disorders. The objective of this study was to investigate the handwriting performance in ADHD and comorbid ADHD-LD adolescents. The study examined the Chinese and English handwriting performance and sensorimotor skills of 32 ADHD, 12 ADHD-LD, and their matched controls. Participants with ADHD had comparable writing time and speed, but the readability was lower than their controls. Participants with ADHD-LD had lower writing speeds in both Chinese and English handwriting than their controls. The ADHD and ADHD-LD groups also showed larger variations in either speed or pen pressure than their controls. Chinese handwriting assessment effectively classified ADHD and ADHD-LD with good sensitivity and positive predictive value. Clinicians should be aware of the fundamental difference between the 2 disorders and make good use of handwriting assessment as a reference to deliver effective therapies and trainings. Copyright © 2018 John Wiley & Sons, Ltd.

Density control in ITER: an iterative learning control and robust control approach

NASA Astrophysics Data System (ADS)

Ravensbergen, T.; de Vries, P. C.; Felici, F.; Blanken, T. C.; Nouailletas, R.; Zabeo, L.

2018-01-01

Plasma density control for next generation tokamaks, such as ITER, is challenging because of multiple reasons. The response of the usual gas valve actuators in future, larger fusion devices, might be too slow for feedback control. Both pellet fuelling and the use of feedforward-based control may help to solve this problem. Also, tight density limits arise during ramp-up, due to operational limits related to divertor detachment and radiative collapses. As the number of shots available for controller tuning will be limited in ITER, in this paper, iterative learning control (ILC) is proposed to determine optimal feedforward actuator inputs based on tracking errors, obtained in previous shots. This control method can take the actuator and density limits into account and can deal with large actuator delays. However, a purely feedforward-based density control may not be sufficient due to the presence of disturbances and shot-to-shot differences. Therefore, robust control synthesis is used to construct a robustly stabilizing feedback controller. In simulations, it is shown that this combined controller strategy is able to achieve good tracking performance in the presence of shot-to-shot differences, tight constraints, and model mismatches.

Fuzzy Adaptive Control for Intelligent Autonomous Space Exploration Problems

NASA Technical Reports Server (NTRS)

Esogbue, Augustine O.

1998-01-01

The principal objective of the research reported here is the re-design, analysis and optimization of our newly developed neural network fuzzy adaptive controller model for complex processes capable of learning fuzzy control rules using process data and improving its control through on-line adaption. The learned improvement is according to a performance objective function that provides evaluative feedback; this performance objective is broadly defined to meet long-range goals over time. Although fuzzy control had proven effective for complex, nonlinear, imprecisely-defined processes for which standard models and controls are either inefficient, impractical or cannot be derived, the state of the art prior to our work showed that procedures for deriving fuzzy control, however, were mostly ad hoc heuristics. The learning ability of neural networks was exploited to systematically derive fuzzy control and permit on-line adaption and in the process optimize control. The operation of neural networks integrates very naturally with fuzzy logic. The neural networks which were designed and tested using simulation software and simulated data, followed by realistic industrial data were reconfigured for application on several platforms as well as for the employment of improved algorithms. The statistical procedures of the learning process were investigated and evaluated with standard statistical procedures (such as ANOVA, graphical analysis of residuals, etc.). The computational advantage of dynamic programming-like methods of optimal control was used to permit on-line fuzzy adaptive control. Tests for the consistency, completeness and interaction of the control rules were applied. Comparisons to other methods and controllers were made so as to identify the major advantages of the resulting controller model. Several specific modifications and extensions were made to the original controller. Additional modifications and explorations have been proposed for further study. Some of these are in progress in our laboratory while others await additional support. All of these enhancements will improve the attractiveness of the controller as an effective tool for the on line control of an array of complex process environments.

Optical implementation of neural learning algorithms based on cross-gain modulation in a semiconductor optical amplifier

NASA Astrophysics Data System (ADS)

Li, Qiang; Wang, Zhi; Le, Yansi; Sun, Chonghui; Song, Xiaojia; Wu, Chongqing

2016-10-01

Neuromorphic engineering has a wide range of applications in the fields of machine learning, pattern recognition, adaptive control, etc. Photonics, characterized by its high speed, wide bandwidth, low power consumption and massive parallelism, is an ideal way to realize ultrafast spiking neural networks (SNNs). Synaptic plasticity is believed to be critical for learning, memory and development in neural circuits. Experimental results have shown that changes of synapse are highly dependent on the relative timing of pre- and postsynaptic spikes. Synaptic plasticity in which presynaptic spikes preceding postsynaptic spikes results in strengthening, while the opposite timing results in weakening is called antisymmetric spike-timing-dependent plasticity (STDP) learning rule. And synaptic plasticity has the opposite effect under the same conditions is called antisymmetric anti-STDP learning rule. We proposed and experimentally demonstrated an optical implementation of neural learning algorithms, which can achieve both of antisymmetric STDP and anti-STDP learning rule, based on the cross-gain modulation (XGM) within a single semiconductor optical amplifier (SOA). The weight and height of the potentitation and depression window can be controlled by adjusting the injection current of the SOA, to mimic the biological antisymmetric STDP and anti-STDP learning rule more realistically. As the injection current increases, the width of depression and potentitation window decreases and height increases, due to the decreasing of recovery time and increasing of gain under a stronger injection current. Based on the demonstrated optical STDP circuit, ultrafast learning in optical SNNs can be realized.

Inference of genetic network of Xenopus frog egg: improved genetic algorithm.

PubMed

Wu, Shinq-Jen; Chou, Chia-Hsien; Wu, Cheng-Tao; Lee, Tsu-Tian

2006-01-01

An improved genetic algorithm (IGA) is proposed to achieve S-system gene network modeling of Xenopus frog egg. Via the time-courses training datasets from Michaelis-Menten model, the optimal parameters are learned. The S-system can clearly describe activative and inhibitory interaction between genes as generating and consuming process. We concern the mitotic control in cell-cycle of Xenopus frog egg to realize cyclin-Cdc2 and Cdc25 for MPF activity. The proposed IGA can achieve global search with migration and keep the best chromosome with elitism operation. The generated gene regulatory networks can provide biological researchers for further experiments in Xenopus frog egg cell cycle control.

Real-Time Smart Grids Control for Preventing Cascading Failures and Blackout using Neural Networks: Experimental Approach for N-1-1 Contingency

NASA Astrophysics Data System (ADS)

Zarrabian, Sina; Belkacemi, Rabie; Babalola, Adeniyi A.

2016-12-01

In this paper, a novel intelligent control is proposed based on Artificial Neural Networks (ANN) to mitigate cascading failure (CF) and prevent blackout in smart grid systems after N-1-1 contingency condition in real-time. The fundamental contribution of this research is to deploy the machine learning concept for preventing blackout at early stages of its occurrence and to make smart grids more resilient, reliable, and robust. The proposed method provides the best action selection strategy for adaptive adjustment of generators' output power through frequency control. This method is able to relieve congestion of transmission lines and prevent consecutive transmission line outage after N-1-1 contingency condition. The proposed ANN-based control approach is tested on an experimental 100 kW test system developed by the authors to test intelligent systems. Additionally, the proposed approach is validated on the large-scale IEEE 118-bus power system by simulation studies. Experimental results show that the ANN approach is very promising and provides accurate and robust control by preventing blackout. The technique is compared to a heuristic multi-agent system (MAS) approach based on communication interchanges. The ANN approach showed more accurate and robust response than the MAS algorithm.

Common Neural Mechanisms Underlying Reversal Learning by Reward and Punishment

PubMed Central

Xue, Gui; Xue, Feng; Droutman, Vita; Lu, Zhong-Lin; Bechara, Antoine; Read, Stephen

2013-01-01

Impairments in flexible goal-directed decisions, often examined by reversal learning, are associated with behavioral abnormalities characterized by impulsiveness and disinhibition. Although the lateral orbital frontal cortex (OFC) has been consistently implicated in reversal learning, it is still unclear whether this region is involved in negative feedback processing, behavioral control, or both, and whether reward and punishment might have different effects on lateral OFC involvement. Using a relatively large sample (N = 47), and a categorical learning task with either monetary reward or moderate electric shock as feedback, we found overlapping activations in the right lateral OFC (and adjacent insula) for reward and punishment reversal learning when comparing correct reversal trials with correct acquisition trials, whereas we found overlapping activations in the right dorsolateral prefrontal cortex (DLPFC) when negative feedback signaled contingency change. The right lateral OFC and DLPFC also showed greater sensitivity to punishment than did their left homologues, indicating an asymmetry in how punishment is processed. We propose that the right lateral OFC and anterior insula are important for transforming affective feedback to behavioral adjustment, whereas the right DLPFC is involved in higher level attention control. These results provide insight into the neural mechanisms of reversal learning and behavioral flexibility, which can be leveraged to understand risky behaviors among vulnerable populations. PMID:24349211

Common neural mechanisms underlying reversal learning by reward and punishment.

PubMed

Xue, Gui; Xue, Feng; Droutman, Vita; Lu, Zhong-Lin; Bechara, Antoine; Read, Stephen

2013-01-01

Impairments in flexible goal-directed decisions, often examined by reversal learning, are associated with behavioral abnormalities characterized by impulsiveness and disinhibition. Although the lateral orbital frontal cortex (OFC) has been consistently implicated in reversal learning, it is still unclear whether this region is involved in negative feedback processing, behavioral control, or both, and whether reward and punishment might have different effects on lateral OFC involvement. Using a relatively large sample (N = 47), and a categorical learning task with either monetary reward or moderate electric shock as feedback, we found overlapping activations in the right lateral OFC (and adjacent insula) for reward and punishment reversal learning when comparing correct reversal trials with correct acquisition trials, whereas we found overlapping activations in the right dorsolateral prefrontal cortex (DLPFC) when negative feedback signaled contingency change. The right lateral OFC and DLPFC also showed greater sensitivity to punishment than did their left homologues, indicating an asymmetry in how punishment is processed. We propose that the right lateral OFC and anterior insula are important for transforming affective feedback to behavioral adjustment, whereas the right DLPFC is involved in higher level attention control. These results provide insight into the neural mechanisms of reversal learning and behavioral flexibility, which can be leveraged to understand risky behaviors among vulnerable populations.

Incentive-Rewarding Mechanism for User-position Control in Mobile Services

NASA Astrophysics Data System (ADS)

Yoshino, Makoto; Sato, Kenichiro; Shinkuma, Ryoichi; Takahashi, Tatsuro

When the number of users in a service area increases in mobile multimedia services, no individual user can obtain satisfactory radio resources such as bandwidth and signal power because the resources are limited and shared. A solution for such a problem is user-position control. In the user-position control, the operator informs users of better communication areas (or spots) and navigates them to these positions. However, because of subjective costs caused by subjects moving from their original to a new position, they do not always attempt to move. To motivate users to contribute their resources in network services that require resource contributions for users, incentive-rewarding mechanisms have been proposed. However, there are no mechanisms that distribute rewards appropriately according to various subjective factors involving users. Furthermore, since the conventional mechanisms limit how rewards are paid, they are applicable only for the network service they targeted. In this paper, we propose a novel incentive-rewarding mechanism to solve these problems, using an external evaluator and interactive learning agents. We also investigated ways of appropriately controlling rewards based on user contributions and system service quality. We applied the proposed mechanism and reward control to the user-position control, and demonstrated its validity.

Flipped-Class Pedagogy Enhances Student Metacognition and Collaborative-Learning Strategies in Higher Education But Effect Does Not Persist

PubMed Central

van Vliet, E. A.; Winnips, J. C.; Brouwer, N.

2015-01-01

In flipped-class pedagogy, students prepare themselves at home before lectures, often by watching short video clips of the course contents. The aim of this study was to investigate the effects of flipped classes on motivation and learning strategies in higher education using a controlled, pre- and posttest approach. The same students were followed in a traditional course and in a course in which flipped classes were substituted for part of the traditional lectures. On the basis of the validated Motivated Strategies for Learning Questionnaire (MSLQ), we found that flipped-class pedagogy enhanced the MSLQ components critical thinking, task value, and peer learning. However, the effects of flipped classes were not long-lasting. We therefore propose repeated use of flipped classes in a curriculum to make effects on metacognition and collaborative-learning strategies sustainable. PMID:26113628

Motivation to Learn and Diversity Training: Application of the Theory of Planned Behavior

ERIC Educational Resources Information Center

Wiethoff, Carolyn

2004-01-01

Although training programs are an important component in most companies' diversity initiatives, little theoretical guidance is available for their implementation. This article proposes a model based on the theory of planned behavior, which addresses the roles of attitude, subjective norms, and perceived behavioral control in motivation to learn…

A machine learning approach to improve contactless heart rate monitoring using a webcam.

PubMed

Monkaresi, Hamed; Calvo, Rafael A; Yan, Hong

2014-07-01

Unobtrusive, contactless recordings of physiological signals are very important for many health and human-computer interaction applications. Most current systems require sensors which intrusively touch the user's skin. Recent advances in contact-free physiological signals open the door to many new types of applications. This technology promises to measure heart rate (HR) and respiration using video only. The effectiveness of this technology, its limitations, and ways of overcoming them deserves particular attention. In this paper, we evaluate this technique for measuring HR in a controlled situation, in a naturalistic computer interaction session, and in an exercise situation. For comparison, HR was measured simultaneously using an electrocardiography device during all sessions. The results replicated the published results in controlled situations, but show that they cannot yet be considered as a valid measure of HR in naturalistic human-computer interaction. We propose a machine learning approach to improve the accuracy of HR detection in naturalistic measurements. The results demonstrate that the root mean squared error is reduced from 43.76 to 3.64 beats/min using the proposed method.

An integrated utility-based model of conflict evaluation and resolution in the Stroop task.

PubMed

Chuderski, Adam; Smolen, Tomasz

2016-04-01

Cognitive control allows humans to direct and coordinate their thoughts and actions in a flexible way, in order to reach internal goals regardless of interference and distraction. The hallmark test used to examine cognitive control is the Stroop task, which elicits both the weakly learned but goal-relevant and the strongly learned but goal-irrelevant response tendencies, and requires people to follow the former while ignoring the latter. After reviewing the existing computational models of cognitive control in the Stroop task, its novel, integrated utility-based model is proposed. The model uses 3 crucial control mechanisms: response utility reinforcement learning, utility-based conflict evaluation using the Festinger formula for assessing the conflict level, and top-down adaptation of response utility in service of conflict resolution. Their complex, dynamic interaction led to replication of 18 experimental effects, being the largest data set explained to date by 1 Stroop model. The simulations cover the basic congruency effects (including the response latency distributions), performance dynamics and adaptation (including EEG indices of conflict), as well as the effects resulting from manipulations applied to stimulation and responding, which are yielded by the extant Stroop literature. (c) 2016 APA, all rights reserved).

Perception-action map learning in controlled multiscroll systems applied to robot navigation.

PubMed

Arena, Paolo; De Fiore, Sebastiano; Fortuna, Luigi; Patané, Luca

2008-12-01

In this paper a new technique for action-oriented perception in robots is presented. The paper starts from exploiting the successful implementation of the basic idea that perceptual states can be embedded into chaotic attractors whose dynamical evolution can be associated with sensorial stimuli. In this way, it can be possible to encode, into the chaotic dynamics, environment-dependent patterns. These have to be suitably linked to an action, executed by the robot, to fulfill an assigned mission. This task is addressed here: the action-oriented perception loop is closed by introducing a simple unsupervised learning stage, implemented via a bio-inspired structure based on the motor map paradigm. In this way, perceptual meanings, useful for solving a given task, can be autonomously learned, based on the environment-dependent patterns embedded into the controlled chaotic dynamics. The presented framework has been tested on a simulated robot and the performance have been successfully compared with other traditional navigation control paradigms. Moreover an implementation of the proposed architecture on a Field Programmable Gate Array is briefly outlined and preliminary experimental results on a roving robot are also reported.

Beyond adaptive-critic creative learning for intelligent mobile robots

NASA Astrophysics Data System (ADS)

Liao, Xiaoqun; Cao, Ming; Hall, Ernest L.

2001-10-01

Intelligent industrial and mobile robots may be considered proven technology in structured environments. Teach programming and supervised learning methods permit solutions to a variety of applications. However, we believe that to extend the operation of these machines to more unstructured environments requires a new learning method. Both unsupervised learning and reinforcement learning are potential candidates for these new tasks. The adaptive critic method has been shown to provide useful approximations or even optimal control policies to non-linear systems. The purpose of this paper is to explore the use of new learning methods that goes beyond the adaptive critic method for unstructured environments. The adaptive critic is a form of reinforcement learning. A critic element provides only high level grading corrections to a cognition module that controls the action module. In the proposed system the critic's grades are modeled and forecasted, so that an anticipated set of sub-grades are available to the cognition model. The forecasting grades are interpolated and are available on the time scale needed by the action model. The success of the system is highly dependent on the accuracy of the forecasted grades and adaptability of the action module. Examples from the guidance of a mobile robot are provided to illustrate the method for simple line following and for the more complex navigation and control in an unstructured environment. The theory presented that is beyond the adaptive critic may be called creative theory. Creative theory is a form of learning that models the highest level of human learning - imagination. The application of the creative theory appears to not only be to mobile robots but also to many other forms of human endeavor such as educational learning and business forecasting. Reinforcement learning such as the adaptive critic may be applied to known problems to aid in the discovery of their solutions. The significance of creative theory is that it permits the discovery of the unknown problems, ones that are not yet recognized but may be critical to survival or success.

Data-driven methods towards learning the highly nonlinear inverse kinematics of tendon-driven surgical manipulators.

PubMed

Xu, Wenjun; Chen, Jie; Lau, Henry Y K; Ren, Hongliang

2017-09-01

Accurate motion control of flexible surgical manipulators is crucial in tissue manipulation tasks. The tendon-driven serpentine manipulator (TSM) is one of the most widely adopted flexible mechanisms in minimally invasive surgery because of its enhanced maneuverability in torturous environments. TSM, however, exhibits high nonlinearities and conventional analytical kinematics model is insufficient to achieve high accuracy. To account for the system nonlinearities, we applied a data driven approach to encode the system inverse kinematics. Three regression methods: extreme learning machine (ELM), Gaussian mixture regression (GMR) and K-nearest neighbors regression (KNNR) were implemented to learn a nonlinear mapping from the robot 3D position states to the control inputs. The performance of the three algorithms was evaluated both in simulation and physical trajectory tracking experiments. KNNR performed the best in the tracking experiments, with the lowest RMSE of 2.1275 mm. The proposed inverse kinematics learning methods provide an alternative and efficient way to accurately model the tendon driven flexible manipulator. Copyright © 2016 John Wiley & Sons, Ltd.

Manifold regularized multitask learning for semi-supervised multilabel image classification.

PubMed

Luo, Yong; Tao, Dacheng; Geng, Bo; Xu, Chao; Maybank, Stephen J

2013-02-01

It is a significant challenge to classify images with multiple labels by using only a small number of labeled samples. One option is to learn a binary classifier for each label and use manifold regularization to improve the classification performance by exploring the underlying geometric structure of the data distribution. However, such an approach does not perform well in practice when images from multiple concepts are represented by high-dimensional visual features. Thus, manifold regularization is insufficient to control the model complexity. In this paper, we propose a manifold regularized multitask learning (MRMTL) algorithm. MRMTL learns a discriminative subspace shared by multiple classification tasks by exploiting the common structure of these tasks. It effectively controls the model complexity because different tasks limit one another's search volume, and the manifold regularization ensures that the functions in the shared hypothesis space are smooth along the data manifold. We conduct extensive experiments, on the PASCAL VOC'07 dataset with 20 classes and the MIR dataset with 38 classes, by comparing MRMTL with popular image classification algorithms. The results suggest that MRMTL is effective for image classification.

Generating Adaptive Behaviour within a Memory-Prediction Framework

PubMed Central

Rawlinson, David; Kowadlo, Gideon

2012-01-01

The Memory-Prediction Framework (MPF) and its Hierarchical-Temporal Memory implementation (HTM) have been widely applied to unsupervised learning problems, for both classification and prediction. To date, there has been no attempt to incorporate MPF/HTM in reinforcement learning or other adaptive systems; that is, to use knowledge embodied within the hierarchy to control a system, or to generate behaviour for an agent. This problem is interesting because the human neocortex is believed to play a vital role in the generation of behaviour, and the MPF is a model of the human neocortex. We propose some simple and biologically-plausible enhancements to the Memory-Prediction Framework. These cause it to explore and interact with an external world, while trying to maximize a continuous, time-varying reward function. All behaviour is generated and controlled within the MPF hierarchy. The hierarchy develops from a random initial configuration by interaction with the world and reinforcement learning only. Among other demonstrations, we show that a 2-node hierarchy can learn to successfully play “rocks, paper, scissors” against a predictable opponent. PMID:22272231

Adaptive Constrained Optimal Control Design for Data-Based Nonlinear Discrete-Time Systems With Critic-Only Structure.

PubMed

Luo, Biao; Liu, Derong; Wu, Huai-Ning

2018-06-01

Reinforcement learning has proved to be a powerful tool to solve optimal control problems over the past few years. However, the data-based constrained optimal control problem of nonaffine nonlinear discrete-time systems has rarely been studied yet. To solve this problem, an adaptive optimal control approach is developed by using the value iteration-based Q-learning (VIQL) with the critic-only structure. Most of the existing constrained control methods require the use of a certain performance index and only suit for linear or affine nonlinear systems, which is unreasonable in practice. To overcome this problem, the system transformation is first introduced with the general performance index. Then, the constrained optimal control problem is converted to an unconstrained optimal control problem. By introducing the action-state value function, i.e., Q-function, the VIQL algorithm is proposed to learn the optimal Q-function of the data-based unconstrained optimal control problem. The convergence results of the VIQL algorithm are established with an easy-to-realize initial condition . To implement the VIQL algorithm, the critic-only structure is developed, where only one neural network is required to approximate the Q-function. The converged Q-function obtained from the critic-only VIQL method is employed to design the adaptive constrained optimal controller based on the gradient descent scheme. Finally, the effectiveness of the developed adaptive control method is tested on three examples with computer simulation.

Event-Triggered Distributed Approximate Optimal State and Output Control of Affine Nonlinear Interconnected Systems.

PubMed

Narayanan, Vignesh; Jagannathan, Sarangapani

2017-06-08

This paper presents an approximate optimal distributed control scheme for a known interconnected system composed of input affine nonlinear subsystems using event-triggered state and output feedback via a novel hybrid learning scheme. First, the cost function for the overall system is redefined as the sum of cost functions of individual subsystems. A distributed optimal control policy for the interconnected system is developed using the optimal value function of each subsystem. To generate the optimal control policy, forward-in-time, neural networks are employed to reconstruct the unknown optimal value function at each subsystem online. In order to retain the advantages of event-triggered feedback for an adaptive optimal controller, a novel hybrid learning scheme is proposed to reduce the convergence time for the learning algorithm. The development is based on the observation that, in the event-triggered feedback, the sampling instants are dynamic and results in variable interevent time. To relax the requirement of entire state measurements, an extended nonlinear observer is designed at each subsystem to recover the system internal states from the measurable feedback. Using a Lyapunov-based analysis, it is demonstrated that the system states and the observer errors remain locally uniformly ultimately bounded and the control policy converges to a neighborhood of the optimal policy. Simulation results are presented to demonstrate the performance of the developed controller.

The emotive nature of conflict monitoring in the medial prefrontal cortex.

PubMed

Saunders, Blair; Lin, Hause; Milyavskaya, Marina; Inzlicht, Michael

2017-09-01

The detection of conflict between incompatible impulses, thoughts, and actions is a ubiquitous source of motivation across theories of goal-directed action. In this overview, we explore the hypothesis that conflict is emotive, integrating perspectives from affective science and cognitive neuroscience. Initially, we review evidence suggesting that the mental and biological processes that monitor for information processing conflict-particularly those generated by the anterior midcingulate cortex-track the affective significance of conflict and use this signal to motivate increased control. In this sense, variation in control resembles a form of affect regulation in which control implementation counteracts the aversive experience of conflict. We also highlight emerging evidence proposing that states and dispositions associated with acceptance facilitate control by tuning individuals to the emotive nature of conflict, before proposing avenues for future research, including investigating the role of affect in reinforcement learning and decision making. Copyright © 2017 Elsevier B.V. All rights reserved.

Perineuronal nets and vocal plasticity in songbirds: A proposed mechanism to explain the difference between closed-ended and open-ended learning.

PubMed

Cornez, Gilles; Madison, Farrah N; Van der Linden, Annemie; Cornil, Charlotte; Yoder, Kathleen M; Ball, Gregory F; Balthazart, Jacques

2017-09-01

Perineuronal nets (PNN) are aggregations of chondroitin sulfate proteoglycans surrounding the soma and proximal processes of neurons, mostly GABAergic interneurons expressing parvalbumin. They limit the plasticity of their afferent synaptic connections. In zebra finches PNN develop in an experience-dependent manner in the song control nuclei HVC and RA (nucleus robustus arcopallialis) when young birds crystallize their song. Because songbird species that are open-ended learners tend to recapitulate each year the different phases of song learning until their song crystallizes at the beginning of the breeding season, we tested whether seasonal changes in PNN expression would be found in the song control nuclei of a seasonally breeding species such as the European starling. Only minimal changes in PNN densities and total number of cells surrounded by PNN were detected. However, comparison of the density of PNN and of PNN surrounding parvalbumin-positive cells revealed that these structures are far less numerous in starlings that show extensive adult vocal plasticity, including learning of new songs throughout the year, than in the closed-ended learner zebra finches. Canaries that also display some vocal plasticity across season but were never formally shown to learn new songs in adulthood were intermediate in this respect. Together these data suggest that establishment of PNN around parvalbumin-positive neurons in song control nuclei has diverged during evolution to control the different learning capacities observed in songbird species. This differential expression of PNN in different songbird species could represent a key cellular mechanism mediating species variation between closed-ended and open-ended learning strategies. © 2017 Wiley Periodicals, Inc. Develop Neurobiol 77: 975-994, 2017. © 2017 Wiley Periodicals, Inc.

Bidirectional extreme learning machine for regression problem and its learning effectiveness.

PubMed

Yang, Yimin; Wang, Yaonan; Yuan, Xiaofang

2012-09-01

It is clear that the learning effectiveness and learning speed of neural networks are in general far slower than required, which has been a major bottleneck for many applications. Recently, a simple and efficient learning method, referred to as extreme learning machine (ELM), was proposed by Huang , which has shown that, compared to some conventional methods, the training time of neural networks can be reduced by a thousand times. However, one of the open problems in ELM research is whether the number of hidden nodes can be further reduced without affecting learning effectiveness. This brief proposes a new learning algorithm, called bidirectional extreme learning machine (B-ELM), in which some hidden nodes are not randomly selected. In theory, this algorithm tends to reduce network output error to 0 at an extremely early learning stage. Furthermore, we find a relationship between the network output error and the network output weights in the proposed B-ELM. Simulation results demonstrate that the proposed method can be tens to hundreds of times faster than other incremental ELM algorithms.

Reward-based learning under hardware constraints-using a RISC processor embedded in a neuromorphic substrate.

PubMed

Friedmann, Simon; Frémaux, Nicolas; Schemmel, Johannes; Gerstner, Wulfram; Meier, Karlheinz

2013-01-01

In this study, we propose and analyze in simulations a new, highly flexible method of implementing synaptic plasticity in a wafer-scale, accelerated neuromorphic hardware system. The study focuses on globally modulated STDP, as a special use-case of this method. Flexibility is achieved by embedding a general-purpose processor dedicated to plasticity into the wafer. To evaluate the suitability of the proposed system, we use a reward modulated STDP rule in a spike train learning task. A single layer of neurons is trained to fire at specific points in time with only the reward as feedback. This model is simulated to measure its performance, i.e., the increase in received reward after learning. Using this performance as baseline, we then simulate the model with various constraints imposed by the proposed implementation and compare the performance. The simulated constraints include discretized synaptic weights, a restricted interface between analog synapses and embedded processor, and mismatch of analog circuits. We find that probabilistic updates can increase the performance of low-resolution weights, a simple interface between analog synapses and processor is sufficient for learning, and performance is insensitive to mismatch. Further, we consider communication latency between wafer and the conventional control computer system that is simulating the environment. This latency increases the delay, with which the reward is sent to the embedded processor. Because of the time continuous operation of the analog synapses, delay can cause a deviation of the updates as compared to the not delayed situation. We find that for highly accelerated systems latency has to be kept to a minimum. This study demonstrates the suitability of the proposed implementation to emulate the selected reward modulated STDP learning rule. It is therefore an ideal candidate for implementation in an upgraded version of the wafer-scale system developed within the BrainScaleS project.

Reward-based learning under hardware constraints—using a RISC processor embedded in a neuromorphic substrate

PubMed Central

Friedmann, Simon; Frémaux, Nicolas; Schemmel, Johannes; Gerstner, Wulfram; Meier, Karlheinz

2013-01-01

In this study, we propose and analyze in simulations a new, highly flexible method of implementing synaptic plasticity in a wafer-scale, accelerated neuromorphic hardware system. The study focuses on globally modulated STDP, as a special use-case of this method. Flexibility is achieved by embedding a general-purpose processor dedicated to plasticity into the wafer. To evaluate the suitability of the proposed system, we use a reward modulated STDP rule in a spike train learning task. A single layer of neurons is trained to fire at specific points in time with only the reward as feedback. This model is simulated to measure its performance, i.e., the increase in received reward after learning. Using this performance as baseline, we then simulate the model with various constraints imposed by the proposed implementation and compare the performance. The simulated constraints include discretized synaptic weights, a restricted interface between analog synapses and embedded processor, and mismatch of analog circuits. We find that probabilistic updates can increase the performance of low-resolution weights, a simple interface between analog synapses and processor is sufficient for learning, and performance is insensitive to mismatch. Further, we consider communication latency between wafer and the conventional control computer system that is simulating the environment. This latency increases the delay, with which the reward is sent to the embedded processor. Because of the time continuous operation of the analog synapses, delay can cause a deviation of the updates as compared to the not delayed situation. We find that for highly accelerated systems latency has to be kept to a minimum. This study demonstrates the suitability of the proposed implementation to emulate the selected reward modulated STDP learning rule. It is therefore an ideal candidate for implementation in an upgraded version of the wafer-scale system developed within the BrainScaleS project. PMID:24065877

Neural control and transient analysis of the LCL-type resonant converter

NASA Astrophysics Data System (ADS)

Zouggar, S.; Nait Charif, H.; Azizi, M.

2000-07-01

This paper proposes a generalised inverse learning structure to control the LCL converter. A feedforward neural network is trained to act as an inverse model of the LCL converter then both are cascaded such that the composed system results in an identity mapping between desired response and the LCL output voltage. Using the large signal model, we analyse the transient output response of the controlled LCL converter in the case of large variation of the load. The simulation results show the efficiency of using neural networks to regulate the LCL converter.

Applying flow chemistry: methods, materials, and multistep synthesis.

PubMed

McQuade, D Tyler; Seeberger, Peter H

2013-07-05

The synthesis of complex molecules requires control over both chemical reactivity and reaction conditions. While reactivity drives the majority of chemical discovery, advances in reaction condition control have accelerated method development/discovery. Recent tools include automated synthesizers and flow reactors. In this Synopsis, we describe how flow reactors have enabled chemical advances in our groups in the areas of single-stage reactions, materials synthesis, and multistep reactions. In each section, we detail the lessons learned and propose future directions.

Max-margin multiattribute learning with low-rank constraint.

PubMed

Zhang, Qiang; Chen, Lin; Li, Baoxin

2014-07-01

Attribute learning has attracted a lot of interests in recent years for its advantage of being able to model high-level concepts with a compact set of midlevel attributes. Real-world objects often demand multiple attributes for effective modeling. Most existing methods learn attributes independently without explicitly considering their intrinsic relatedness. In this paper, we propose max margin multiattribute learning with low-rank constraint, which learns a set of attributes simultaneously, using only relative ranking of the attributes for the data. By learning all the attributes simultaneously through low-rank constraint, the proposed method is able to capture their intrinsic correlation for improved learning; by requiring only relative ranking, the method avoids restrictive binary labels of attributes that are often assumed by many existing techniques. The proposed method is evaluated on both synthetic data and real visual data including a challenging video data set. Experimental results demonstrate the effectiveness of the proposed method.

Sleep-dependent memory consolidation and accelerated forgetting

PubMed Central

Atherton, Kathryn E.; Nobre, Anna C.; Zeman, Adam Z.; Butler, Christopher R.

2014-01-01

Accelerated long-term forgetting (ALF) is a form of memory impairment in which learning and initial retention of information appear normal but subsequent forgetting is excessively rapid. ALF is most commonly associated with epilepsy and, in particular, a form of late-onset epilepsy called transient epileptic amnesia (TEA). ALF provides a novel opportunity to investigate post-encoding memory processes, such as consolidation. Sleep is implicated in the consolidation of memory in healthy people and a deficit in sleep-dependent memory consolidation has been proposed as an explanation for ALF. If this proposal were correct, then sleep would not benefit memory retention in people with ALF as much as in healthy people, and ALF might only be apparent when the retention interval contains sleep. To test this theory, we compared performance on a sleep-sensitive memory task over a night of sleep and a day of wakefulness. We found, contrary to the hypothesis, that sleep benefits memory retention in TEA patients with ALF and that this benefit is no smaller in magnitude than that seen in healthy controls. Indeed, the patients performed significantly more poorly than the controls only in the wake condition and not the sleep condition. Patients were matched to controls on learning rate, initial retention, and the effect of time of day on cognitive performance. These results indicate that ALF is not caused by a disruption of sleep-dependent memory consolidation. Instead, ALF may be due to an encoding abnormality that goes undetected on behavioural assessments of learning, or by a deficit in memory consolidation processes that are not sleep-dependent. PMID:24657478

Reasoning, learning, and creativity: frontal lobe function and human decision-making.

PubMed

Collins, Anne; Koechlin, Etienne

2012-01-01

The frontal lobes subserve decision-making and executive control--that is, the selection and coordination of goal-directed behaviors. Current models of frontal executive function, however, do not explain human decision-making in everyday environments featuring uncertain, changing, and especially open-ended situations. Here, we propose a computational model of human executive function that clarifies this issue. Using behavioral experiments, we show that unlike others, the proposed model predicts human decisions and their variations across individuals in naturalistic situations. The model reveals that for driving action, the human frontal function monitors up to three/four concurrent behavioral strategies and infers online their ability to predict action outcomes: whenever one appears more reliable than unreliable, this strategy is chosen to guide the selection and learning of actions that maximize rewards. Otherwise, a new behavioral strategy is tentatively formed, partly from those stored in long-term memory, then probed, and if competitive confirmed to subsequently drive action. Thus, the human executive function has a monitoring capacity limited to three or four behavioral strategies. This limitation is compensated by the binary structure of executive control that in ambiguous and unknown situations promotes the exploration and creation of new behavioral strategies. The results support a model of human frontal function that integrates reasoning, learning, and creative abilities in the service of decision-making and adaptive behavior.

Design and Implementation of a Smart LED Lighting System Using a Self Adaptive Weighted Data Fusion Algorithm

PubMed Central

Sung, Wen-Tsai; Lin, Jia-Syun

2013-01-01

This work aims to develop a smart LED lighting system, which is remotely controlled by Android apps via handheld devices, e.g., smartphones, tablets, and so forth. The status of energy use is reflected by readings displayed on a handheld device, and it is treated as a criterion in the lighting mode design of a system. A multimeter, a wireless light dimmer, an IR learning remote module, etc. are connected to a server by means of RS 232/485 and a human computer interface on a touch screen. The wireless data communication is designed to operate in compliance with the ZigBee standard, and signal processing on sensed data is made through a self adaptive weighted data fusion algorithm. A low variation in data fusion together with a high stability is experimentally demonstrated in this work. The wireless light dimmer as well as the IR learning remote module can be instructed directly by command given on the human computer interface, and the reading on a multimeter can be displayed thereon via the server. This proposed smart LED lighting system can be remotely controlled and self learning mode can be enabled by a single handheld device via WiFi transmission. Hence, this proposal is validated as an approach to power monitoring for home appliances, and is demonstrated as a digital home network in consideration of energy efficiency.

Neuroscientific Model of Motivational Process

PubMed Central

Kim, Sung-il

2013-01-01

Considering the neuroscientific findings on reward, learning, value, decision-making, and cognitive control, motivation can be parsed into three sub processes, a process of generating motivation, a process of maintaining motivation, and a process of regulating motivation. I propose a tentative neuroscientific model of motivational processes which consists of three distinct but continuous sub processes, namely reward-driven approach, value-based decision-making, and goal-directed control. Reward-driven approach is the process in which motivation is generated by reward anticipation and selective approach behaviors toward reward. This process recruits the ventral striatum (reward area) in which basic stimulus-action association is formed, and is classified as an automatic motivation to which relatively less attention is assigned. By contrast, value-based decision-making is the process of evaluating various outcomes of actions, learning through positive prediction error, and calculating the value continuously. The striatum and the orbitofrontal cortex (valuation area) play crucial roles in sustaining motivation. Lastly, the goal-directed control is the process of regulating motivation through cognitive control to achieve goals. This consciously controlled motivation is associated with higher-level cognitive functions such as planning, retaining the goal, monitoring the performance, and regulating action. The anterior cingulate cortex (attention area) and the dorsolateral prefrontal cortex (cognitive control area) are the main neural circuits related to regulation of motivation. These three sub processes interact with each other by sending reward prediction error signals through dopaminergic pathway from the striatum and to the prefrontal cortex. The neuroscientific model of motivational process suggests several educational implications with regard to the generation, maintenance, and regulation of motivation to learn in the learning environment. PMID:23459598

Neuroscientific model of motivational process.

PubMed

Kim, Sung-Il

2013-01-01

Considering the neuroscientific findings on reward, learning, value, decision-making, and cognitive control, motivation can be parsed into three sub processes, a process of generating motivation, a process of maintaining motivation, and a process of regulating motivation. I propose a tentative neuroscientific model of motivational processes which consists of three distinct but continuous sub processes, namely reward-driven approach, value-based decision-making, and goal-directed control. Reward-driven approach is the process in which motivation is generated by reward anticipation and selective approach behaviors toward reward. This process recruits the ventral striatum (reward area) in which basic stimulus-action association is formed, and is classified as an automatic motivation to which relatively less attention is assigned. By contrast, value-based decision-making is the process of evaluating various outcomes of actions, learning through positive prediction error, and calculating the value continuously. The striatum and the orbitofrontal cortex (valuation area) play crucial roles in sustaining motivation. Lastly, the goal-directed control is the process of regulating motivation through cognitive control to achieve goals. This consciously controlled motivation is associated with higher-level cognitive functions such as planning, retaining the goal, monitoring the performance, and regulating action. The anterior cingulate cortex (attention area) and the dorsolateral prefrontal cortex (cognitive control area) are the main neural circuits related to regulation of motivation. These three sub processes interact with each other by sending reward prediction error signals through dopaminergic pathway from the striatum and to the prefrontal cortex. The neuroscientific model of motivational process suggests several educational implications with regard to the generation, maintenance, and regulation of motivation to learn in the learning environment.

Adaptive two-degree-of-freedom PI for speed control of permanent magnet synchronous motor based on fractional order GPC.

PubMed

Qiao, Wenjun; Tang, Xiaoqi; Zheng, Shiqi; Xie, Yuanlong; Song, Bao

2016-09-01

In this paper, an adaptive two-degree-of-freedom (2Dof) proportional-integral (PI) controller is proposed for the speed control of permanent magnet synchronous motor (PMSM). Firstly, an enhanced just-in-time learning technique consisting of two novel searching engines is presented to identify the model of the speed control system in a real-time manner. Secondly, a general formula is given to predict the future speed reference which is unavailable at the interval of two bus-communication cycles. Thirdly, the fractional order generalized predictive control (FOGPC) is introduced to improve the control performance of the servo drive system. Based on the identified model parameters and predicted speed reference, the optimal control law of FOGPC is derived. Finally, the designed 2Dof PI controller is auto-tuned by matching with the optimal control law. Simulations and real-time experimental results on the servo drive system of PMSM are provided to illustrate the effectiveness of the proposed strategy. Copyright © 2016 ISA. Published by Elsevier Ltd. All rights reserved.

Adaptive Filter Design Using Type-2 Fuzzy Cerebellar Model Articulation Controller.

PubMed

Lin, Chih-Min; Yang, Ming-Shu; Chao, Fei; Hu, Xiao-Min; Zhang, Jun

2016-10-01

This paper aims to propose an efficient network and applies it as an adaptive filter for the signal processing problems. An adaptive filter is proposed using a novel interval type-2 fuzzy cerebellar model articulation controller (T2FCMAC). The T2FCMAC realizes an interval type-2 fuzzy logic system based on the structure of the CMAC. Due to the better ability of handling uncertainties, type-2 fuzzy sets can solve some complicated problems with outstanding effectiveness than type-1 fuzzy sets. In addition, the Lyapunov function is utilized to derive the conditions of the adaptive learning rates, so that the convergence of the filtering error can be guaranteed. In order to demonstrate the performance of the proposed adaptive T2FCMAC filter, it is tested in signal processing applications, including a nonlinear channel equalization system, a time-varying channel equalization system, and an adaptive noise cancellation system. The advantages of the proposed filter over the other adaptive filters are verified through simulations.

Robust tracking and distributed synchronization control of a multi-motor servomechanism with H-infinity performance.

PubMed

Wang, Minlin; Ren, Xuemei; Chen, Qiang

2018-01-01

The multi-motor servomechanism (MMS) is a multi-variable, high coupling and nonlinear system, which makes the controller design challenging. In this paper, an adaptive robust H-infinity control scheme is proposed to achieve both the load tracking and multi-motor synchronization of MMS. This control scheme consists of two parts: a robust tracking controller and a distributed synchronization controller. The robust tracking controller is constructed by incorporating a neural network (NN) K-filter observer into the dynamic surface control, while the distributed synchronization controller is designed by combining the mean deviation coupling control strategy with the distributed technique. The proposed control scheme has several merits: 1) by using the mean deviation coupling synchronization control strategy, the tracking controller and the synchronization controller can be designed individually without any coupling problem; 2) the immeasurable states and unknown nonlinearities are handled by a NN K-filter observer, where the number of NN weights is largely reduced by using the minimal learning parameter technique; 3) the H-infinity performances of tracking error and synchronization error are guaranteed by introducing a robust term into the tracking controller and the synchronization controller, respectively. The stabilities of the tracking and synchronization control systems are analyzed by the Lyapunov theory. Simulation and experimental results based on a four-motor servomechanism are conducted to demonstrate the effectiveness of the proposed method. Copyright © 2017 ISA. Published by Elsevier Ltd. All rights reserved.

Optimized Assistive Human-Robot Interaction Using Reinforcement Learning.

PubMed

Modares, Hamidreza; Ranatunga, Isura; Lewis, Frank L; Popa, Dan O

2016-03-01

An intelligent human-robot interaction (HRI) system with adjustable robot behavior is presented. The proposed HRI system assists the human operator to perform a given task with minimum workload demands and optimizes the overall human-robot system performance. Motivated by human factor studies, the presented control structure consists of two control loops. First, a robot-specific neuro-adaptive controller is designed in the inner loop to make the unknown nonlinear robot behave like a prescribed robot impedance model as perceived by a human operator. In contrast to existing neural network and adaptive impedance-based control methods, no information of the task performance or the prescribed robot impedance model parameters is required in the inner loop. Then, a task-specific outer-loop controller is designed to find the optimal parameters of the prescribed robot impedance model to adjust the robot's dynamics to the operator skills and minimize the tracking error. The outer loop includes the human operator, the robot, and the task performance details. The problem of finding the optimal parameters of the prescribed robot impedance model is transformed into a linear quadratic regulator (LQR) problem which minimizes the human effort and optimizes the closed-loop behavior of the HRI system for a given task. To obviate the requirement of the knowledge of the human model, integral reinforcement learning is used to solve the given LQR problem. Simulation results on an x - y table and a robot arm, and experimental implementation results on a PR2 robot confirm the suitability of the proposed method.

Spiral and Project-Based Learning with Peer Assessment in a Computer Science Project Management Course

ERIC Educational Resources Information Center

Jaime, Arturo; Blanco, José Miguel; Domínguez, César; Sánchez, Ana; Heras, Jónathan; Usandizaga, Imanol

2016-01-01

Different learning methods such as project-based learning, spiral learning and peer assessment have been implemented in science disciplines with different outcomes. This paper presents a proposal for a project management course in the context of a computer science degree. Our proposal combines three well-known methods: project-based learning,…

Spike-timing dependent inhibitory plasticity to learn a selective gating of backpropagating action potentials.

PubMed

Wilmes, Katharina Anna; Schleimer, Jan-Hendrik; Schreiber, Susanne

2017-04-01

Inhibition is known to influence the forward-directed flow of information within neurons. However, also regulation of backward-directed signals, such as backpropagating action potentials (bAPs), can enrich the functional repertoire of local circuits. Inhibitory control of bAP spread, for example, can provide a switch for the plasticity of excitatory synapses. Although such a mechanism is possible, it requires a precise timing of inhibition to annihilate bAPs without impairment of forward-directed excitatory information flow. Here, we propose a specific learning rule for inhibitory synapses to automatically generate the correct timing to gate bAPs in pyramidal cells when embedded in a local circuit of feedforward inhibition. Based on computational modeling of multi-compartmental neurons with physiological properties, we demonstrate that a learning rule with anti-Hebbian shape can establish the required temporal precision. In contrast to classical spike-timing dependent plasticity of excitatory synapses, the proposed inhibitory learning mechanism does not necessarily require the definition of an upper bound of synaptic weights because of its tendency to self-terminate once annihilation of bAPs has been reached. Our study provides a functional context in which one of the many time-dependent learning rules that have been observed experimentally - specifically, a learning rule with anti-Hebbian shape - is assigned a relevant role for inhibitory synapses. Moreover, the described mechanism is compatible with an upregulation of excitatory plasticity by disinhibition. © 2016 Federation of European Neuroscience Societies and John Wiley & Sons Ltd.

Approximated affine projection algorithm for feedback cancellation in hearing aids.

PubMed

Lee, Sangmin; Kim, In-Young; Park, Young-Cheol

2007-09-01

We propose an approximated affine projection (AP) algorithm for feedback cancellation in hearing aids. It is based on the conventional approach using the Gauss-Seidel (GS) iteration, but provides more stable convergence behaviour even with small step sizes. In the proposed algorithm, a residue of the weighted error vector, instead of the current error sample, is used to provide stable convergence. A new learning rate control scheme is also applied to the proposed algorithm to prevent signal cancellation and system instability. The new scheme determines step size in proportion to the prediction factor of the input, so that adaptation is inhibited whenever tone-like signals are present in the input. Simulation results verified the efficiency of the proposed algorithm.

An Improved Incremental Learning Approach for KPI Prognosis of Dynamic Fuel Cell System.

PubMed

Yin, Shen; Xie, Xiaochen; Lam, James; Cheung, Kie Chung; Gao, Huijun

2016-12-01

The key performance indicator (KPI) has an important practical value with respect to the product quality and economic benefits for modern industry. To cope with the KPI prognosis issue under nonlinear conditions, this paper presents an improved incremental learning approach based on available process measurements. The proposed approach takes advantage of the algorithm overlapping of locally weighted projection regression (LWPR) and partial least squares (PLS), implementing the PLS-based prognosis in each locally linear model produced by the incremental learning process of LWPR. The global prognosis results including KPI prediction and process monitoring are obtained from the corresponding normalized weighted means of all the local models. The statistical indicators for prognosis are enhanced as well by the design of novel KPI-related and KPI-unrelated statistics with suitable control limits for non-Gaussian data. For application-oriented purpose, the process measurements from real datasets of a proton exchange membrane fuel cell system are employed to demonstrate the effectiveness of KPI prognosis. The proposed approach is finally extended to a long-term voltage prediction for potential reference of further fuel cell applications.

Providing Controlled Exposure To Target Vocabulary through the Screening and Arranging of Texts.

ERIC Educational Resources Information Center

Ghadirian, Sina

2002-01-01

Considers how to bring foreign language students with limited vocabulary knowledge, consisting mainly of high-frequency words, to where they can comprehend authentic texts in a target domain. Proposes bridging the vocabulary gap by determining which word families account for 95% of the target domain's running words, and having students learn these…

Improving Security for SCADA Sensor Networks with Reputation Systems and Self-Organizing Maps.

PubMed

Moya, José M; Araujo, Alvaro; Banković, Zorana; de Goyeneche, Juan-Mariano; Vallejo, Juan Carlos; Malagón, Pedro; Villanueva, Daniel; Fraga, David; Romero, Elena; Blesa, Javier

2009-01-01

The reliable operation of modern infrastructures depends on computerized systems and Supervisory Control and Data Acquisition (SCADA) systems, which are also based on the data obtained from sensor networks. The inherent limitations of the sensor devices make them extremely vulnerable to cyberwarfare/cyberterrorism attacks. In this paper, we propose a reputation system enhanced with distributed agents, based on unsupervised learning algorithms (self-organizing maps), in order to achieve fault tolerance and enhanced resistance to previously unknown attacks. This approach has been extensively simulated and compared with previous proposals.

Improving Security for SCADA Sensor Networks with Reputation Systems and Self-Organizing Maps

PubMed Central

Moya, José M.; Araujo, Álvaro; Banković, Zorana; de Goyeneche, Juan-Mariano; Vallejo, Juan Carlos; Malagón, Pedro; Villanueva, Daniel; Fraga, David; Romero, Elena; Blesa, Javier

2009-01-01

The reliable operation of modern infrastructures depends on computerized systems and Supervisory Control and Data Acquisition (SCADA) systems, which are also based on the data obtained from sensor networks. The inherent limitations of the sensor devices make them extremely vulnerable to cyberwarfare/cyberterrorism attacks. In this paper, we propose a reputation system enhanced with distributed agents, based on unsupervised learning algorithms (self-organizing maps), in order to achieve fault tolerance and enhanced resistance to previously unknown attacks. This approach has been extensively simulated and compared with previous proposals. PMID:22291569

Multi Agent Systems with Symbiotic Learning and Evolution using GNP

NASA Astrophysics Data System (ADS)

Eguchi, Toru; Hirasawa, Kotaro; Hu, Jinglu; Murata, Junichi

Recently, various attempts relevant to Multi Agent Systems (MAS) which is one of the most promising systems based on Distributed Artificial Intelligence have been studied to control large and complicated systems efficiently. In these trends of MAS, Multi Agent Systems with Symbiotic Learning and Evolution named Masbiole has been proposed. In Masbiole, symbiotic phenomena among creatures are considered in the process of learning and evolution of MAS. So we can expect more flexible and sophisticated solutions than conventional MAS. In this paper, we apply Masbiole to Iterative Prisoner’s Dilemma Games (IPD Games) using Genetic Network Programming (GNP) which is a newly developed evolutionary computation method for constituting agents. Some characteristics of Masbiole using GNP in IPD Games are clarified.

Hierarchical HMM based learning of navigation primitives for cooperative robotic endovascular catheterization.

PubMed

Rafii-Tari, Hedyeh; Liu, Jindong; Payne, Christopher J; Bicknell, Colin; Yang, Guang-Zhong

2014-01-01

Despite increased use of remote-controlled steerable catheter navigation systems for endovascular intervention, most current designs are based on master configurations which tend to alter natural operator tool interactions. This introduces problems to both ergonomics and shared human-robot control. This paper proposes a novel cooperative robotic catheterization system based on learning-from-demonstration. By encoding the higher-level structure of a catheterization task as a sequence of primitive motions, we demonstrate how to achieve prospective learning for complex tasks whilst incorporating subject-specific variations. A hierarchical Hidden Markov Model is used to model each movement primitive as well as their sequential relationship. This model is applied to generation of motion sequences, recognition of operator input, and prediction of future movements for the robot. The framework is validated by comparing catheter tip motions against the manual approach, showing significant improvements in the quality of catheterization. The results motivate the design of collaborative robotic systems that are intuitive to use, while reducing the cognitive workload of the operator.

A global bioheat model with self-tuning optimal regulation of body temperature using Hebbian feedback covariance learning.

PubMed

Ong, M L; Ng, E Y K

2005-12-01

In the lower brain, body temperature is continually being regulated almost flawlessly despite huge fluctuations in ambient and physiological conditions that constantly threaten the well-being of the body. The underlying control problem defining thermal homeostasis is one of great enormity: Many systems and sub-systems are involved in temperature regulation and physiological processes are intrinsically complex and intertwined. Thus the defining control system has to take into account the complications of nonlinearities, system uncertainties, delayed feedback loops as well as internal and external disturbances. In this paper, we propose a self-tuning adaptive thermal controller based upon Hebbian feedback covariance learning where the system is to be regulated continually to best suit its environment. This hypothesis is supported in part by postulations of the presence of adaptive optimization behavior in biological systems of certain organisms which face limited resources vital for survival. We demonstrate the use of Hebbian feedback covariance learning as a possible self-adaptive controller in body temperature regulation. The model postulates an important role of Hebbian covariance adaptation as a means of reinforcement learning in the thermal controller. The passive system is based on a simplified 2-node core and shell representation of the body, where global responses are captured. Model predictions are consistent with observed thermoregulatory responses to conditions of exercise and rest, and heat and cold stress. An important implication of the model is that optimal physiological behaviors arising from self-tuning adaptive regulation in the thermal controller may be responsible for the departure from homeostasis in abnormal states, e.g., fever. This was previously unexplained using the conventional "set-point" control theory.

Encouraging junior community netball players to learn correct safe landing technique.

PubMed

White, Peta E; Ullah, Shahid; Donaldson, Alex; Otago, Leonie; Saunders, Natalie; Romiti, Maria; Finch, Caroline F

2012-01-01

Behavioural factors and beliefs are important determinants of the adoption of sports injury interventions. This study aimed to understand behavioural factors associated with junior community netball players' intentions to learn correct landing technique during coach-led training sessions, proposed as a means of reducing their risk of lower limb injury. Cross-sectional survey. 287 female players from 58 junior netball teams in the 2007/2008-summer competition completed a 13-item questionnaire developed from the Theory of Planned Behaviour (TPB). This assessed players' attitudes (four items), subjective norms (four), perceived behavioural control (four) and intentions (one) around the safety behaviour of learning correct landing technique at netball training. All items were rated on a seven-point bipolar scale. Cluster-adjusted logistic regression was used to assess which TPB constructs were most associated with strong intentions. Players had positive intentions and attitudes towards learning safe landing technique and perceived positive social pressure from significant others. They also perceived themselves to have considerable control over engaging (or not) in this behaviour. Players' attitudes (p<0.001) and subjective norms (p<0.001), but not perceived behavioural control (p=0.49), were associated with strong intentions to learn correct landing technique at training. Injury prevention implementation strategies aimed at maximising junior players' participation in correct landing training programs should emphasise the benefits of learning correct landing technique (i.e. change attitudes) and involve significant others and role models whom junior players admire (i.e. capitalise on social norms) in the promotion of such programs. Copyright © 2011 Sports Medicine Australia. Published by Elsevier Ltd. All rights reserved.

Proposing an Optimal Learning Architecture for the Digital Enterprise.

ERIC Educational Resources Information Center

O'Driscoll, Tony

2003-01-01

Discusses the strategic role of learning in information age organizations; analyzes parallels between the application of technology to business and the application of technology to learning; and proposes a learning architecture that aligns with the knowledge-based view of the firm and optimizes the application of technology to achieve proficiency…

Smart Cruise Control: UAV sensor operator intent estimation and its application

NASA Astrophysics Data System (ADS)

Cheng, Hui; Butler, Darren; Kumar, Rakesh

2006-05-01

Due to their long endurance, superior mobility and the low risk posed to the pilot and sensor operator, UAVs have become the preferred platform for persistent ISR missions. However, currently most UAV based ISR missions are conducted through manual operation. Event the simplest tasks, such as vehicle tracking, route reconnaissance and site monitoring, need the sensor operator's undivided attention and constant adjustment of the sensor control. The lack of autonomous behaviour greatly limits of the effectiveness and the capability of UAV-based ISR, especially the use of a large number of UAVs simultaneously. Although fully autonomous UAV based ISR system is desirable, it is still a distant dream due to the complexity and diversity of combat and ISR missions. In this paper, we propose a Smart Cruise Control system that can learn UAV sensor operator's intent and use it to complete tasks automatically, such as route reconnaissance and site monitoring. Using an operator attention model, the proposed system can estimate the operator's intent from how they control the sensor (e.g. camera) and the content of the imagery that is acquired. Therefore, for example, from initially manually controlling the UAV sensor to follow a road, the system can learn not only the preferred operation, "tracking", but also the road appearance, "what to track" in real-time. Then, the learnt models of both road and the desired operation can be used to complete the task automatically. We have demonstrated the Smart Cruise Control system using real UAV videos where roads need to be tracked and buildings need to be monitored.

Effects of the Digital Game-Development Approach on Elementary School Students' Learning Motivation, Problem Solving, and Learning Achievement

ERIC Educational Resources Information Center

Chu, Hui-Chun; Hung, Chun-Ming

2015-01-01

In this study, the game-based development approach is proposed for improving the learning motivation, problem solving skills, and learning achievement of students. An experiment was conducted on a learning activity of an elementary school science course to evaluate the performance of the proposed approach. A total of 59 sixth graders from two…

Correlation Filter Learning Toward Peak Strength for Visual Tracking.

PubMed

Sui, Yao; Wang, Guanghui; Zhang, Li

2018-04-01

This paper presents a novel visual tracking approach to correlation filter learning toward peak strength of correlation response. Previous methods leverage all features of the target and the immediate background to learn a correlation filter. Some features, however, may be distractive to tracking, like those from occlusion and local deformation, resulting in unstable tracking performance. This paper aims at solving this issue and proposes a novel algorithm to learn the correlation filter. The proposed approach, by imposing an elastic net constraint on the filter, can adaptively eliminate those distractive features in the correlation filtering. A new peak strength metric is proposed to measure the discriminative capability of the learned correlation filter. It is demonstrated that the proposed approach effectively strengthens the peak of the correlation response, leading to more discriminative performance than previous methods. Extensive experiments on a challenging visual tracking benchmark demonstrate that the proposed tracker outperforms most state-of-the-art methods.

A Local Learning Rule for Independent Component Analysis

PubMed Central

Isomura, Takuya; Toyoizumi, Taro

2016-01-01

Humans can separately recognize independent sources when they sense their superposition. This decomposition is mathematically formulated as independent component analysis (ICA). While a few biologically plausible learning rules, so-called local learning rules, have been proposed to achieve ICA, their performance varies depending on the parameters characterizing the mixed signals. Here, we propose a new learning rule that is both easy to implement and reliable. Both mathematical and numerical analyses confirm that the proposed rule outperforms other local learning rules over a wide range of parameters. Notably, unlike other rules, the proposed rule can separate independent sources without any preprocessing, even if the number of sources is unknown. The successful performance of the proposed rule is then demonstrated using natural images and movies. We discuss the implications of this finding for our understanding of neuronal information processing and its promising applications to neuromorphic engineering. PMID:27323661

Training trajectories by continuous recurrent multilayer networks.

PubMed

Leistritz, L; Galicki, M; Witte, H; Kochs, E

2002-01-01

This paper addresses the problem of training trajectories by means of continuous recurrent neural networks whose feedforward parts are multilayer perceptrons. Such networks can approximate a general nonlinear dynamic system with arbitrary accuracy. The learning process is transformed into an optimal control framework where the weights are the controls to be determined. A training algorithm based upon a variational formulation of Pontryagin's maximum principle is proposed for such networks. Computer examples demonstrating the efficiency of the given approach are also presented.

Deep imitation learning for 3D navigation tasks.

PubMed

Hussein, Ahmed; Elyan, Eyad; Gaber, Mohamed Medhat; Jayne, Chrisina

2018-01-01

Deep learning techniques have shown success in learning from raw high-dimensional data in various applications. While deep reinforcement learning is recently gaining popularity as a method to train intelligent agents, utilizing deep learning in imitation learning has been scarcely explored. Imitation learning can be an efficient method to teach intelligent agents by providing a set of demonstrations to learn from. However, generalizing to situations that are not represented in the demonstrations can be challenging, especially in 3D environments. In this paper, we propose a deep imitation learning method to learn navigation tasks from demonstrations in a 3D environment. The supervised policy is refined using active learning in order to generalize to unseen situations. This approach is compared to two popular deep reinforcement learning techniques: deep-Q-networks and Asynchronous actor-critic (A3C). The proposed method as well as the reinforcement learning methods employ deep convolutional neural networks and learn directly from raw visual input. Methods for combining learning from demonstrations and experience are also investigated. This combination aims to join the generalization ability of learning by experience with the efficiency of learning by imitation. The proposed methods are evaluated on 4 navigation tasks in a 3D simulated environment. Navigation tasks are a typical problem that is relevant to many real applications. They pose the challenge of requiring demonstrations of long trajectories to reach the target and only providing delayed rewards (usually terminal) to the agent. The experiments show that the proposed method can successfully learn navigation tasks from raw visual input while learning from experience methods fail to learn an effective policy. Moreover, it is shown that active learning can significantly improve the performance of the initially learned policy using a small number of active samples.

Distance learning in academic health education.

PubMed

Mattheos, N; Schittek, M; Attström, R; Lyon, H C

2001-05-01

Distance learning is an apparent alternative to traditional methods in education of health care professionals. Non-interactive distance learning, interactive courses and virtual learning environments exist as three different generations in distance learning, each with unique methodologies, strengths and potential. Different methodologies have been recommended for distance learning, varying from a didactic approach to a problem-based learning procedure. Accreditation, teamwork and personal contact between the tutors and the students during a course provided by distance learning are recommended as motivating factors in order to enhance the effectiveness of the learning. Numerous assessment methods for distance learning courses have been proposed. However, few studies report adequate tests for the effectiveness of the distance-learning environment. Available information indicates that distance learning may significantly decrease the cost of academic health education at all levels. Furthermore, such courses can provide education to students and professionals not accessible by traditional methods. Distance learning applications still lack the support of a solid theoretical framework and are only evaluated to a limited extent. Cases reported so far tend to present enthusiastic results, while more carefully-controlled studies suggest a cautious attitude towards distance learning. There is a vital need for research evidence to identify the factors of importance and variables involved in distance learning. The effectiveness of distance learning courses, especially in relation to traditional teaching methods, must therefore be further investigated.

Quantum-enhanced deliberation of learning agents using trapped ions

NASA Astrophysics Data System (ADS)

Dunjko, V.; Friis, N.; Briegel, H. J.

2015-02-01

A scheme that successfully employs quantum mechanics in the design of autonomous learning agents has recently been reported in the context of the projective simulation (PS) model for artificial intelligence. In that approach, the key feature of a PS agent, a specific type of memory which is explored via random walks, was shown to be amenable to quantization, allowing for a speed-up. In this work we propose an implementation of such classical and quantum agents in systems of trapped ions. We employ a generic construction by which the classical agents are ‘upgraded’ to their quantum counterparts by a nested process of adding coherent control, and we outline how this construction can be realized in ion traps. Our results provide a flexible modular architecture for the design of PS agents. Furthermore, we present numerical simulations of simple PS agents which analyze the robustness of our proposal under certain noise models.

Learning new gait patterns: Exploratory muscle activity during motor learning is not predicted by motor modules

PubMed Central

Ranganathan, Rajiv; Krishnan, Chandramouli; Dhaher, Yasin Y.; Rymer, William Z.

2018-01-01

The motor module hypothesis in motor control proposes that the nervous system can simplify the problem of controlling a large number of muscles in human movement by grouping muscles into a smaller number of modules. Here, we tested one prediction of the modular organization hypothesis by examining whether there is preferential exploration along these motor modules during the learning of a new gait pattern. Healthy college-aged participants learned a new gait pattern which required increased hip and knee flexion during the swing phase while walking in a lower-extremity robot (Lokomat). The new gait pattern was displayed as a foot trajectory in the sagittal plane and participants attempted to match their foot trajectory to this template. We recorded EMG from 8 lower-extremity muscles and we extracted motor modules during both baseline walking and target-tracking using non-negative matrix factorization (NMF). Results showed increased trajectory variability in the first block of learning, indicating that participants were engaged in exploratory behavior. Critically, when we examined the muscle activity during this exploratory phase, we found that the composition of motor modules changed significantly within the first few strides of attempting the new gait pattern. The lack of persistence of the motor modules under even short time scales suggests that motor modules extracted during locomotion may be more indicative of correlated muscle activity induced by the task constraints of walking, rather than reflecting a modular control strategy. PMID:26916510

Cognitive Control Predicts Use of Model-Based Reinforcement-Learning

PubMed Central

Otto, A. Ross; Skatova, Anya; Madlon-Kay, Seth; Daw, Nathaniel D.

2015-01-01

Accounts of decision-making and its neural substrates have long posited the operation of separate, competing valuation systems in the control of choice behavior. Recent theoretical and experimental work suggest that this classic distinction between behaviorally and neurally dissociable systems for habitual and goal-directed (or more generally, automatic and controlled) choice may arise from two computational strategies for reinforcement learning (RL), called model-free and model-based RL, but the cognitive or computational processes by which one system may dominate over the other in the control of behavior is a matter of ongoing investigation. To elucidate this question, we leverage the theoretical framework of cognitive control, demonstrating that individual differences in utilization of goal-related contextual information—in the service of overcoming habitual, stimulus-driven responses—in established cognitive control paradigms predict model-based behavior in a separate, sequential choice task. The behavioral correspondence between cognitive control and model-based RL compellingly suggests that a common set of processes may underpin the two behaviors. In particular, computational mechanisms originally proposed to underlie controlled behavior may be applicable to understanding the interactions between model-based and model-free choice behavior. PMID:25170791

Artificial intelligent e-learning architecture

NASA Astrophysics Data System (ADS)

Alharbi, Mafawez; Jemmali, Mahdi

2017-03-01

Many institutions and university has forced to use e learning, due to its ability to provide additional and flexible solutions for students and researchers. E-learning In the last decade have transported about the extreme changes in the distribution of education allowing learners to access multimedia course material at any time, from anywhere to suit their specific needs. In the form of e learning, instructors and learners live in different places and they do not engage in a classroom environment, but within virtual universe. Many researches have defined e learning based on their objectives. Therefore, there are small number of e-learning architecture have proposed in the literature. However, the proposed architecture has lack of embedding intelligent system in the architecture of e learning. This research argues that unexplored potential remains, as there is scope for e learning to be intelligent system. This research proposes e-learning architecture that incorporates intelligent system. There are intelligence components, which built into the architecture.

Feedback control stabilization of critical dynamics via resource transport on multilayer networks: How glia enable learning dynamics in the brain

NASA Astrophysics Data System (ADS)

Virkar, Yogesh S.; Shew, Woodrow L.; Restrepo, Juan G.; Ott, Edward

2016-10-01

Learning and memory are acquired through long-lasting changes in synapses. In the simplest models, such synaptic potentiation typically leads to runaway excitation, but in reality there must exist processes that robustly preserve overall stability of the neural system dynamics. How is this accomplished? Various approaches to this basic question have been considered. Here we propose a particularly compelling and natural mechanism for preserving stability of learning neural systems. This mechanism is based on the global processes by which metabolic resources are distributed to the neurons by glial cells. Specifically, we introduce and study a model composed of two interacting networks: a model neural network interconnected by synapses that undergo spike-timing-dependent plasticity; and a model glial network interconnected by gap junctions that diffusively transport metabolic resources among the glia and, ultimately, to neural synapses where they are consumed. Our main result is that the biophysical constraints imposed by diffusive transport of metabolic resources through the glial network can prevent runaway growth of synaptic strength, both during ongoing activity and during learning. Our findings suggest a previously unappreciated role for glial transport of metabolites in the feedback control stabilization of neural network dynamics during learning.

Collective learning for the emergence of social norms in networked multiagent systems.

PubMed

Yu, Chao; Zhang, Minjie; Ren, Fenghui

2014-12-01

Social norms such as social rules and conventions play a pivotal role in sustaining system order by regulating and controlling individual behaviors toward a global consensus in large-scale distributed systems. Systematic studies of efficient mechanisms that can facilitate the emergence of social norms enable us to build and design robust distributed systems, such as electronic institutions and norm-governed sensor networks. This paper studies the emergence of social norms via learning from repeated local interactions in networked multiagent systems. A collective learning framework, which imitates the opinion aggregation process in human decision making, is proposed to study the impact of agent local collective behaviors on the emergence of social norms in a number of different situations. In the framework, each agent interacts repeatedly with all of its neighbors. At each step, an agent first takes a best-response action toward each of its neighbors and then combines all of these actions into a final action using ensemble learning methods. Extensive experiments are carried out to evaluate the framework with respect to different network topologies, learning strategies, numbers of actions, influences of nonlearning agents, and so on. Experimental results reveal some significant insights into the manipulation and control of norm emergence in networked multiagent systems achieved through local collective behaviors.

Infectious disease, behavioural flexibility and the evolution of culture in primates.

PubMed

McCabe, Collin M; Reader, Simon M; Nunn, Charles L

2015-01-22

Culturally transmitted traits are observed in a wide array of animal species, yet we understand little about the costs of the behavioural patterns that underlie culture, such as innovation and social learning. We propose that infectious diseases are a significant cost associated with cultural transmission. We investigated two hypotheses that may explain such a connection: that social learning and exploratory behaviours (specifically, innovation and extractive foraging) either compensate for existing infection or increase exposure to infectious agents. We used Bayesian comparative methods, controlling for sampling effort, body mass, group size, geographical range size, terrestriality, latitude and phylogenetic uncertainty. Across 127 primate species, we found a positive association between pathogen richness and rates of innovation, extractive foraging and social learning. This relationship was driven by two independent phenomena: socially contagious diseases were positively associated with rates of social learning, and environmentally transmitted diseases were positively associated with rates of exploration. Because higher pathogen burdens can contribute to morbidity and mortality, we propose that parasitism is a significant cost associated with the behavioural patterns that underpin culture, and that increased pathogen exposure is likely to have played an important role in the evolution of culture in both non-human primates and humans. © 2014 The Author(s) Published by the Royal Society. All rights reserved.

Wavefront cellular learning automata.

PubMed

Moradabadi, Behnaz; Meybodi, Mohammad Reza

2018-02-01

This paper proposes a new cellular learning automaton, called a wavefront cellular learning automaton (WCLA). The proposed WCLA has a set of learning automata mapped to a connected structure and uses this structure to propagate the state changes of the learning automata over the structure using waves. In the WCLA, after one learning automaton chooses its action, if this chosen action is different from the previous action, it can send a wave to its neighbors and activate them. Each neighbor receiving the wave is activated and must choose a new action. This structure for the WCLA is necessary in many dynamic areas such as social networks, computer networks, grid computing, and web mining. In this paper, we introduce the WCLA framework as an optimization tool with diffusion capability, study its behavior over time using ordinary differential equation solutions, and present its accuracy using expediency analysis. To show the superiority of the proposed WCLA, we compare the proposed method with some other types of cellular learning automata using two benchmark problems.

Wavefront cellular learning automata

NASA Astrophysics Data System (ADS)

Moradabadi, Behnaz; Meybodi, Mohammad Reza

2018-02-01

This paper proposes a new cellular learning automaton, called a wavefront cellular learning automaton (WCLA). The proposed WCLA has a set of learning automata mapped to a connected structure and uses this structure to propagate the state changes of the learning automata over the structure using waves. In the WCLA, after one learning automaton chooses its action, if this chosen action is different from the previous action, it can send a wave to its neighbors and activate them. Each neighbor receiving the wave is activated and must choose a new action. This structure for the WCLA is necessary in many dynamic areas such as social networks, computer networks, grid computing, and web mining. In this paper, we introduce the WCLA framework as an optimization tool with diffusion capability, study its behavior over time using ordinary differential equation solutions, and present its accuracy using expediency analysis. To show the superiority of the proposed WCLA, we compare the proposed method with some other types of cellular learning automata using two benchmark problems.

A Novel Percutaneous Electrode Implant for Improving Robustness in Advanced Myoelectric Control

PubMed Central

Hahne, Janne M.; Farina, Dario; Jiang, Ning; Liebetanz, David

2016-01-01

Despite several decades of research, electrically powered hand and arm prostheses are still controlled with very simple algorithms that process the surface electromyogram (EMG) of remnant muscles to achieve control of one prosthetic function at a time. More advanced machine learning methods have shown promising results under laboratory conditions. However, limited robustness has largely prevented the transfer of these laboratory advances to clinical applications. In this paper, we introduce a novel percutaneous EMG electrode to be implanted chronically with the aim of improving the reliability of EMG detection in myoelectric control. The proposed electrode requires a minimally invasive procedure for its implantation, similar to a cosmetic micro-dermal implant. Moreover, being percutaneous, it does not require power and data telemetry modules. Four of these electrodes were chronically implanted in the forearm of an able-bodied human volunteer for testing their characteristics. The implants showed significantly lower impedance and greater robustness against mechanical interference than traditional surface EMG electrodes used for myoelectric control. Moreover, the EMG signals detected by the proposed systems allowed more stable control performance across sessions in different days than that achieved with classic EMG electrodes. In conclusion, the proposed implants may be a promising interface for clinically available prostheses. PMID:27065783

The transition to increased automaticity during finger sequence learning in adult males who stutter.

PubMed

Smits-Bandstra, Sarah; De Nil, Luc; Rochon, Elizabeth

2006-01-01

The present study compared the automaticity levels of persons who stutter (PWS) and persons who do not stutter (PNS) on a practiced finger sequencing task under dual task conditions. Automaticity was defined as the amount of attention required for task performance. Twelve PWS and 12 control subjects practiced finger tapping sequences under single and then dual task conditions. Control subjects performed the sequencing task significantly faster and less variably under single versus dual task conditions while PWS' performance was consistently slow and variable (comparable to the dual task performance of control subjects) under both conditions. Control subjects were significantly more accurate on a colour recognition distracter task than PWS under dual task conditions. These results suggested that control subjects transitioned to quick, accurate and increasingly automatic performance on the sequencing task after practice, while PWS did not. Because most stuttering treatment programs for adults include practice and automatization of new motor speech skills, findings of this finger sequencing study and future studies of speech sequence learning may have important implications for how to maximize stuttering treatment effectiveness. As a result of this activity, the participant will be able to: (1) Define automaticity and explain the importance of dual task paradigms to investigate automaticity; (2) Relate the proposed relationship between motor learning and automaticity as stated by the authors; (3) Summarize the reviewed literature concerning the performance of PWS on dual tasks; and (4) Explain why the ability to transition to automaticity during motor learning may have important clinical implications for stuttering treatment effectiveness.

The Proposed Model of Collaborative Virtual Learning Environment for Introductory Programming Course

ERIC Educational Resources Information Center

Othman, Mahfudzah; Othman, Muhaini

2012-01-01

This paper discusses the proposed model of the collaborative virtual learning system for the introductory computer programming course which uses one of the collaborative learning techniques known as the "Think-Pair-Share". The main objective of this study is to design a model for an online learning system that facilitates the…

E-Learning System for Design and Construction of Amplifier Using Transistors

ERIC Educational Resources Information Center

Takemura, Atsushi

2014-01-01

This paper proposes a novel e-Learning system for the comprehensive understanding of electronic circuits with transistors. The proposed e-Learning system allows users to learn a wide range of topics, encompassing circuit theories, design, construction, and measurement. Given the fact that the amplifiers with transistors are an integral part of…

Evaluating a Proposed Learning Experience in Terms of Eight Learning Theories.

ERIC Educational Resources Information Center

Abram, Marie J.

The work of eight learning theorists was used to evaluate a proposed adult education/learning experience in an effort to operationalize a system for locating strengths and weaknesses in an instructional system prior to its implementation. Thirty-five implications for adult education were extrapolated from work representing the Behaviorist (B.F.…

An Augmented Reality-Based Mobile Learning System to Improve Students' Learning Achievements and Motivations in Natural Science Inquiry Activities

ERIC Educational Resources Information Center

Chiang, Tosti H. C.; Yang, Stephen J. H.; Hwang, Gwo-Jen

2014-01-01

In this study, an augmented reality-based mobile learning system is proposed for conducting inquiry-based learning activities. An experiment has been conducted to examine the effectiveness of the proposed approach in terms of learning achievements and motivations. The subjects were 57 fourth graders from two classes taught by the same teacher in…

Co-Creating a Culturally Responsive Distance Education Cancer Course with, and for, Alaska's Community Health Workers: Motivations from a Survey of Key Stakeholders.

PubMed

Cueva, Katie; Revels, Laura; Kuhnley, Regina; Cueva, Melany; Lanier, Anne; Dignan, Mark

2017-09-01

To gain input on a proposed culturally responsive, distance-delivered cancer education course informed by empowerment theory and adult-learning principles, Alaska's Community Health Aides/Practitioners (CHA/Ps) and CHA/P leadership were invited to take an online survey in February 2015. The proposed course will be developed as part of the "Distance Education to Engage Alaskan Community Health Aides in Cancer Control" project. The results of the survey demonstrate that respondents are both interested in taking the proposed class and engaging in course development. The results also indicate that respondents have the technological comfort and capacity to be engaged in online learning and have primarily positive experiences and perceptions of distance education. This survey is the beginning of the interactive development of the online cancer education course and part of a continuing endeavor to promote wellness with, and for, Alaska's people by empowering Alaska's CHA/Ps and inspiring positive behavioral change to both prevent cancer and support those who feel its burdens.

The power of associative learning and the ontogeny of optimal behaviour.

PubMed

Enquist, Magnus; Lind, Johan; Ghirlanda, Stefano

2016-11-01

Behaving efficiently (optimally or near-optimally) is central to animals' adaptation to their environment. Much evolutionary biology assumes, implicitly or explicitly, that optimal behavioural strategies are genetically inherited, yet the behaviour of many animals depends crucially on learning. The question of how learning contributes to optimal behaviour is largely open. Here we propose an associative learning model that can learn optimal behaviour in a wide variety of ecologically relevant circumstances. The model learns through chaining, a term introduced by Skinner to indicate learning of behaviour sequences by linking together shorter sequences or single behaviours. Our model formalizes the concept of conditioned reinforcement (the learning process that underlies chaining) and is closely related to optimization algorithms from machine learning. Our analysis dispels the common belief that associative learning is too limited to produce 'intelligent' behaviour such as tool use, social learning, self-control or expectations of the future. Furthermore, the model readily accounts for both instinctual and learned aspects of behaviour, clarifying how genetic evolution and individual learning complement each other, and bridging a long-standing divide between ethology and psychology. We conclude that associative learning, supported by genetic predispositions and including the oft-neglected phenomenon of conditioned reinforcement, may suffice to explain the ontogeny of optimal behaviour in most, if not all, non-human animals. Our results establish associative learning as a more powerful optimizing mechanism than acknowledged by current opinion.

The power of associative learning and the ontogeny of optimal behaviour

PubMed Central

Enquist, Magnus; Lind, Johan

2016-01-01

Behaving efficiently (optimally or near-optimally) is central to animals' adaptation to their environment. Much evolutionary biology assumes, implicitly or explicitly, that optimal behavioural strategies are genetically inherited, yet the behaviour of many animals depends crucially on learning. The question of how learning contributes to optimal behaviour is largely open. Here we propose an associative learning model that can learn optimal behaviour in a wide variety of ecologically relevant circumstances. The model learns through chaining, a term introduced by Skinner to indicate learning of behaviour sequences by linking together shorter sequences or single behaviours. Our model formalizes the concept of conditioned reinforcement (the learning process that underlies chaining) and is closely related to optimization algorithms from machine learning. Our analysis dispels the common belief that associative learning is too limited to produce ‘intelligent’ behaviour such as tool use, social learning, self-control or expectations of the future. Furthermore, the model readily accounts for both instinctual and learned aspects of behaviour, clarifying how genetic evolution and individual learning complement each other, and bridging a long-standing divide between ethology and psychology. We conclude that associative learning, supported by genetic predispositions and including the oft-neglected phenomenon of conditioned reinforcement, may suffice to explain the ontogeny of optimal behaviour in most, if not all, non-human animals. Our results establish associative learning as a more powerful optimizing mechanism than acknowledged by current opinion. PMID:28018662

Two-layer contractive encodings for learning stable nonlinear features.

PubMed

Schulz, Hannes; Cho, Kyunghyun; Raiko, Tapani; Behnke, Sven

2015-04-01

Unsupervised learning of feature hierarchies is often a good strategy to initialize deep architectures for supervised learning. Most existing deep learning methods build these feature hierarchies layer by layer in a greedy fashion using either auto-encoders or restricted Boltzmann machines. Both yield encoders which compute linear projections of input followed by a smooth thresholding function. In this work, we demonstrate that these encoders fail to find stable features when the required computation is in the exclusive-or class. To overcome this limitation, we propose a two-layer encoder which is less restricted in the type of features it can learn. The proposed encoder is regularized by an extension of previous work on contractive regularization. This proposed two-layer contractive encoder potentially poses a more difficult optimization problem, and we further propose to linearly transform hidden neurons of the encoder to make learning easier. We demonstrate the advantages of the two-layer encoders qualitatively on artificially constructed datasets as well as commonly used benchmark datasets. We also conduct experiments on a semi-supervised learning task and show the benefits of the proposed two-layer encoders trained with the linear transformation of perceptrons. Copyright © 2014 Elsevier Ltd. All rights reserved.

Altered neural encoding of prediction errors in assault-related posttraumatic stress disorder.

PubMed

Ross, Marisa C; Lenow, Jennifer K; Kilts, Clinton D; Cisler, Josh M

2018-05-12

Posttraumatic stress disorder (PTSD) is widely associated with deficits in extinguishing learned fear responses, which relies on mechanisms of reinforcement learning (e.g., updating expectations based on prediction errors). However, the degree to which PTSD is associated with impairments in general reinforcement learning (i.e., outside of the context of fear stimuli) remains poorly understood. Here, we investigate brain and behavioral differences in general reinforcement learning between adult women with and without a current diagnosis of PTSD. 29 adult females (15 PTSD with exposure to assaultive violence, 14 controls) underwent a neutral reinforcement-learning task (i.e., two arm bandit task) during fMRI. We modeled participant behavior using different adaptations of the Rescorla-Wagner (RW) model and used Independent Component Analysis to identify timecourses for large-scale a priori brain networks. We found that an anticorrelated and risk sensitive RW model best fit participant behavior, with no differences in computational parameters between groups. Women in the PTSD group demonstrated significantly less neural encoding of prediction errors in both a ventral striatum/mPFC and anterior insula network compared to healthy controls. Weakened encoding of prediction errors in the ventral striatum/mPFC and anterior insula during a general reinforcement learning task, outside of the context of fear stimuli, suggests the possibility of a broader conceptualization of learning differences in PTSD than currently proposed in current neurocircuitry models of PTSD. Copyright © 2018 Elsevier Ltd. All rights reserved.

Improvements from a flipped classroom may simply be the fruits of active learning.

PubMed

Jensen, Jamie L; Kummer, Tyler A; d M Godoy, Patricia D

2015-03-02

The "flipped classroom" is a learning model in which content attainment is shifted forward to outside of class, then followed by instructor-facilitated concept application activities in class. Current studies on the flipped model are limited. Our goal was to provide quantitative and controlled data about the effectiveness of this model. Using a quasi-experimental design, we compared an active nonflipped classroom with an active flipped classroom, both using the 5-E learning cycle, in an effort to vary only the role of the instructor and control for as many of the other potentially influential variables as possible. Results showed that both low-level and deep conceptual learning were equivalent between the conditions. Attitudinal data revealed equal student satisfaction with the course. Interestingly, both treatments ranked their contact time with the instructor as more influential to their learning than what they did at home. We conclude that the flipped classroom does not result in higher learning gains or better attitudes compared with the nonflipped classroom when both utilize an active-learning, constructivist approach and propose that learning gains in either condition are most likely a result of the active-learning style of instruction rather than the order in which the instructor participated in the learning process. © 2015 J. L. Jensen et al. CBE—Life Sciences Education © 2015 The American Society for Cell Biology. This article is distributed by The American Society for Cell Biology under license from the author(s). It is available to the public under an Attribution–Noncommercial–Share Alike 3.0 Unported Creative Commons License (http://creativecommons.org/licenses/by-nc-sa/3.0).

Deep learning of joint myelin and T1w MRI features in normal-appearing brain tissue to distinguish between multiple sclerosis patients and healthy controls.

PubMed

Yoo, Youngjin; Tang, Lisa Y W; Brosch, Tom; Li, David K B; Kolind, Shannon; Vavasour, Irene; Rauscher, Alexander; MacKay, Alex L; Traboulsee, Anthony; Tam, Roger C

2018-01-01

Myelin imaging is a form of quantitative magnetic resonance imaging (MRI) that measures myelin content and can potentially allow demyelinating diseases such as multiple sclerosis (MS) to be detected earlier. Although focal lesions are the most visible signs of MS pathology on conventional MRI, it has been shown that even tissues that appear normal may exhibit decreased myelin content as revealed by myelin-specific images (i.e., myelin maps). Current methods for analyzing myelin maps typically use global or regional mean myelin measurements to detect abnormalities, but ignore finer spatial patterns that may be characteristic of MS. In this paper, we present a machine learning method to automatically learn, from multimodal MR images, latent spatial features that can potentially improve the detection of MS pathology at early stage. More specifically, 3D image patches are extracted from myelin maps and the corresponding T1-weighted (T1w) MRIs, and are used to learn a latent joint myelin-T1w feature representation via unsupervised deep learning. Using a data set of images from MS patients and healthy controls, a common set of patches are selected via a voxel-wise t -test performed between the two groups. In each MS image, any patches overlapping with focal lesions are excluded, and a feature imputation method is used to fill in the missing values. A feature selection process (LASSO) is then utilized to construct a sparse representation. The resulting normal-appearing features are used to train a random forest classifier. Using the myelin and T1w images of 55 relapse-remitting MS patients and 44 healthy controls in an 11-fold cross-validation experiment, the proposed method achieved an average classification accuracy of 87.9% (SD = 8.4%), which is higher and more consistent across folds than those attained by regional mean myelin (73.7%, SD = 13.7%) and T1w measurements (66.7%, SD = 10.6%), or deep-learned features in either the myelin (83.8%, SD = 11.0%) or T1w (70.1%, SD = 13.6%) images alone, suggesting that the proposed method has strong potential for identifying image features that are more sensitive and specific to MS pathology in normal-appearing brain tissues.

Product Quality Modelling Based on Incremental Support Vector Machine

NASA Astrophysics Data System (ADS)

Wang, J.; Zhang, W.; Qin, B.; Shi, W.

2012-05-01

Incremental Support vector machine (ISVM) is a new learning method developed in recent years based on the foundations of statistical learning theory. It is suitable for the problem of sequentially arriving field data and has been widely used for product quality prediction and production process optimization. However, the traditional ISVM learning does not consider the quality of the incremental data which may contain noise and redundant data; it will affect the learning speed and accuracy to a great extent. In order to improve SVM training speed and accuracy, a modified incremental support vector machine (MISVM) is proposed in this paper. Firstly, the margin vectors are extracted according to the Karush-Kuhn-Tucker (KKT) condition; then the distance from the margin vectors to the final decision hyperplane is calculated to evaluate the importance of margin vectors, where the margin vectors are removed while their distance exceed the specified value; finally, the original SVs and remaining margin vectors are used to update the SVM. The proposed MISVM can not only eliminate the unimportant samples such as noise samples, but also can preserve the important samples. The MISVM has been experimented on two public data and one field data of zinc coating weight in strip hot-dip galvanizing, and the results shows that the proposed method can improve the prediction accuracy and the training speed effectively. Furthermore, it can provide the necessary decision supports and analysis tools for auto control of product quality, and also can extend to other process industries, such as chemical process and manufacturing process.

MULTI-SOURCE FEATURE LEARNING FOR JOINT ANALYSIS OF INCOMPLETE MULTIPLE HETEROGENEOUS NEUROIMAGING DATA

PubMed Central

Yuan, Lei; Wang, Yalin; Thompson, Paul M.; Narayan, Vaibhav A.; Ye, Jieping

2012-01-01

Analysis of incomplete data is a big challenge when integrating large-scale brain imaging datasets from different imaging modalities. In the Alzheimer’s Disease Neuroimaging Initiative (ADNI), for example, over half of the subjects lack cerebrospinal fluid (CSF) measurements; an independent half of the subjects do not have fluorodeoxyglucose positron emission tomography (FDG-PET) scans; many lack proteomics measurements. Traditionally, subjects with missing measures are discarded, resulting in a severe loss of available information. In this paper, we address this problem by proposing an incomplete Multi-Source Feature (iMSF) learning method where all the samples (with at least one available data source) can be used. To illustrate the proposed approach, we classify patients from the ADNI study into groups with Alzheimer’s disease (AD), mild cognitive impairment (MCI) and normal controls, based on the multi-modality data. At baseline, ADNI’s 780 participants (172 AD, 397 MCI, 211 NC), have at least one of four data types: magnetic resonance imaging (MRI), FDG-PET, CSF and proteomics. These data are used to test our algorithm. Depending on the problem being solved, we divide our samples according to the availability of data sources, and we learn shared sets of features with state-of-the-art sparse learning methods. To build a practical and robust system, we construct a classifier ensemble by combining our method with four other methods for missing value estimation. Comprehensive experiments with various parameters show that our proposed iMSF method and the ensemble model yield stable and promising results. PMID:22498655

Output feedback control of a quadrotor UAV using neural networks.

PubMed

Dierks, Travis; Jagannathan, Sarangapani

2010-01-01

In this paper, a new nonlinear controller for a quadrotor unmanned aerial vehicle (UAV) is proposed using neural networks (NNs) and output feedback. The assumption on the availability of UAV dynamics is not always practical, especially in an outdoor environment. Therefore, in this work, an NN is introduced to learn the complete dynamics of the UAV online, including uncertain nonlinear terms like aerodynamic friction and blade flapping. Although a quadrotor UAV is underactuated, a novel NN virtual control input scheme is proposed which allows all six degrees of freedom (DOF) of the UAV to be controlled using only four control inputs. Furthermore, an NN observer is introduced to estimate the translational and angular velocities of the UAV, and an output feedback control law is developed in which only the position and the attitude of the UAV are considered measurable. It is shown using Lyapunov theory that the position, orientation, and velocity tracking errors, the virtual control and observer estimation errors, and the NN weight estimation errors for each NN are all semiglobally uniformly ultimately bounded (SGUUB) in the presence of bounded disturbances and NN functional reconstruction errors while simultaneously relaxing the separation principle. The effectiveness of proposed output feedback control scheme is then demonstrated in the presence of unknown nonlinear dynamics and disturbances, and simulation results are included to demonstrate the theoretical conjecture.

Simultaneous Local Binary Feature Learning and Encoding for Homogeneous and Heterogeneous Face Recognition.

PubMed

Lu, Jiwen; Erin Liong, Venice; Zhou, Jie

2017-08-09

In this paper, we propose a simultaneous local binary feature learning and encoding (SLBFLE) approach for both homogeneous and heterogeneous face recognition. Unlike existing hand-crafted face descriptors such as local binary pattern (LBP) and Gabor features which usually require strong prior knowledge, our SLBFLE is an unsupervised feature learning approach which automatically learns face representation from raw pixels. Unlike existing binary face descriptors such as the LBP, discriminant face descriptor (DFD), and compact binary face descriptor (CBFD) which use a two-stage feature extraction procedure, our SLBFLE jointly learns binary codes and the codebook for local face patches so that discriminative information from raw pixels from face images of different identities can be obtained by using a one-stage feature learning and encoding procedure. Moreover, we propose a coupled simultaneous local binary feature learning and encoding (C-SLBFLE) method to make the proposed approach suitable for heterogeneous face matching. Unlike most existing coupled feature learning methods which learn a pair of transformation matrices for each modality, we exploit both the common and specific information from heterogeneous face samples to characterize their underlying correlations. Experimental results on six widely used face datasets are presented to demonstrate the effectiveness of the proposed method.

Real-Time Decentralized Neural Control via Backstepping for a Robotic Arm Powered by Industrial Servomotors.

PubMed

Vazquez, Luis A; Jurado, Francisco; Castaneda, Carlos E; Santibanez, Victor

2018-02-01

This paper presents a continuous-time decentralized neural control scheme for trajectory tracking of a two degrees of freedom direct drive vertical robotic arm. A decentralized recurrent high-order neural network (RHONN) structure is proposed to identify online, in a series-parallel configuration and using the filtered error learning law, the dynamics of the plant. Based on the RHONN subsystems, a local neural controller is derived via backstepping approach. The effectiveness of the decentralized neural controller is validated on a robotic arm platform, of our own design and unknown parameters, which uses industrial servomotors to drive the joints.

Improving Motor Control in Walking: A Randomized Clinical Trial in Older Adults with Subclinical Walking Difficulty

PubMed Central

Brach, Jennifer S.; Lowry, Kristin; Perera, Subashan; Hornyak, Victoria; Wert, David; Studenski, Stephanie A.; VanSwearingen, Jessie M.

2016-01-01

Objective The objective was to test the proposed mechanism of action of a task-specific motor learning intervention by examining its effect on measures of the motor control of gait. Design Single blinded randomized clinical trial. Setting University research laboratory. Participants Forty older adults 65 years of age and older, with gait speed >1.0 m/s and impaired motor skill (Figure of 8 walk time > 8 secs). Interventions The two interventions included a task-oriented motor learning and a standard exercise program. Both interventions lasted 12 weeks, with twice weekly one hour physical therapist supervised sessions. Main Outcome Measures Two measure of the motor control of gait, gait variability and smoothness of walking, were assessed pre and post intervention by assessors masked to treatment arm. Results Of 40 randomized subjects; 38 completed the trial (mean age 77.1±6.0 years). Motor control group improved more than standard group in double support time variability (0.13 vs. 0.05 m/s; adjusted difference, AD=0.006, p=0.03). Smoothness of walking in the anterior/posterior direction improved more in motor control than standard for all conditions (usual: AD=0.53, p=0.05; narrow: AD=0.56, p=0.01; dual task: AD=0.57, p=0.04). Conclusions Among older adults with subclinical walking difficulty, there is initial evidence that task-oriented motor learning exercise results in gains in the motor control of walking, while standard exercise does not. Task-oriented motor learning exercise is a promising intervention for improving timing and coordination deficits related to mobility difficulties in older adults, and needs to be evaluated in a definitive larger trial. PMID:25448244

Self-Learning Power Control in Wireless Sensor Networks.

PubMed

Chincoli, Michele; Liotta, Antonio

2018-01-27

Current trends in interconnecting myriad smart objects to monetize on Internet of Things applications have led to high-density communications in wireless sensor networks. This aggravates the already over-congested unlicensed radio bands, calling for new mechanisms to improve spectrum management and energy efficiency, such as transmission power control. Existing protocols are based on simplistic heuristics that often approach interference problems (i.e., packet loss, delay and energy waste) by increasing power, leading to detrimental results. The scope of this work is to investigate how machine learning may be used to bring wireless nodes to the lowest possible transmission power level and, in turn, to respect the quality requirements of the overall network. Lowering transmission power has benefits in terms of both energy consumption and interference. We propose a protocol of transmission power control through a reinforcement learning process that we have set in a multi-agent system. The agents are independent learners using the same exploration strategy and reward structure, leading to an overall cooperative network. The simulation results show that the system converges to an equilibrium where each node transmits at the minimum power while respecting high packet reception ratio constraints. Consequently, the system benefits from low energy consumption and packet delay.

Self-Learning Power Control in Wireless Sensor Networks

PubMed Central

Liotta, Antonio

2018-01-01

Current trends in interconnecting myriad smart objects to monetize on Internet of Things applications have led to high-density communications in wireless sensor networks. This aggravates the already over-congested unlicensed radio bands, calling for new mechanisms to improve spectrum management and energy efficiency, such as transmission power control. Existing protocols are based on simplistic heuristics that often approach interference problems (i.e., packet loss, delay and energy waste) by increasing power, leading to detrimental results. The scope of this work is to investigate how machine learning may be used to bring wireless nodes to the lowest possible transmission power level and, in turn, to respect the quality requirements of the overall network. Lowering transmission power has benefits in terms of both energy consumption and interference. We propose a protocol of transmission power control through a reinforcement learning process that we have set in a multi-agent system. The agents are independent learners using the same exploration strategy and reward structure, leading to an overall cooperative network. The simulation results show that the system converges to an equilibrium where each node transmits at the minimum power while respecting high packet reception ratio constraints. Consequently, the system benefits from low energy consumption and packet delay. PMID:29382072

Joint deep shape and appearance learning: application to optic pathway glioma segmentation

NASA Astrophysics Data System (ADS)

Mansoor, Awais; Li, Ien; Packer, Roger J.; Avery, Robert A.; Linguraru, Marius George

2017-03-01

Automated tissue characterization is one of the major applications of computer-aided diagnosis systems. Deep learning techniques have recently demonstrated impressive performance for the image patch-based tissue characterization. However, existing patch-based tissue classification techniques struggle to exploit the useful shape information. Local and global shape knowledge such as the regional boundary changes, diameter, and volumetrics can be useful in classifying the tissues especially in scenarios where the appearance signature does not provide significant classification information. In this work, we present a deep neural network-based method for the automated segmentation of the tumors referred to as optic pathway gliomas (OPG) located within the anterior visual pathway (AVP; optic nerve, chiasm or tracts) using joint shape and appearance learning. Voxel intensity values of commonly used MRI sequences are generally not indicative of OPG. To be considered an OPG, current clinical practice dictates that some portion of AVP must demonstrate shape enlargement. The method proposed in this work integrates multiple sequence magnetic resonance image (T1, T2, and FLAIR) along with local boundary changes to train a deep neural network. For training and evaluation purposes, we used a dataset of multiple sequence MRI obtained from 20 subjects (10 controls, 10 NF1+OPG). To our best knowledge, this is the first deep representation learning-based approach designed to merge shape and multi-channel appearance data for the glioma detection. In our experiments, mean misclassification errors of 2:39% and 0:48% were observed respectively for glioma and control patches extracted from the AVP. Moreover, an overall dice similarity coefficient of 0:87+/-0:13 (0:93+/-0:06 for healthy tissue, 0:78+/-0:18 for glioma tissue) demonstrates the potential of the proposed method in the accurate localization and early detection of OPG.

Approximate dynamic programming for optimal stationary control with control-dependent noise.

PubMed

Jiang, Yu; Jiang, Zhong-Ping

2011-12-01

This brief studies the stochastic optimal control problem via reinforcement learning and approximate/adaptive dynamic programming (ADP). A policy iteration algorithm is derived in the presence of both additive and multiplicative noise using Itô calculus. The expectation of the approximated cost matrix is guaranteed to converge to the solution of some algebraic Riccati equation that gives rise to the optimal cost value. Moreover, the covariance of the approximated cost matrix can be reduced by increasing the length of time interval between two consecutive iterations. Finally, a numerical example is given to illustrate the efficiency of the proposed ADP methodology.

Machine learning methods as a tool to analyse incomplete or irregularly sampled radon time series data.

PubMed

Janik, M; Bossew, P; Kurihara, O

2018-07-15

Machine learning is a class of statistical techniques which has proven to be a powerful tool for modelling the behaviour of complex systems, in which response quantities depend on assumed controls or predictors in a complicated way. In this paper, as our first purpose, we propose the application of machine learning to reconstruct incomplete or irregularly sampled data of time series indoor radon ( 222 Rn). The physical assumption underlying the modelling is that Rn concentration in the air is controlled by environmental variables such as air temperature and pressure. The algorithms "learn" from complete sections of multivariate series, derive a dependence model and apply it to sections where the controls are available, but not the response (Rn), and in this way complete the Rn series. Three machine learning techniques are applied in this study, namely random forest, its extension called the gradient boosting machine and deep learning. For a comparison, we apply the classical multiple regression in a generalized linear model version. Performance of the models is evaluated through different metrics. The performance of the gradient boosting machine is found to be superior to that of the other techniques. By applying learning machines, we show, as our second purpose, that missing data or periods of Rn series data can be reconstructed and resampled on a regular grid reasonably, if data of appropriate physical controls are available. The techniques also identify to which degree the assumed controls contribute to imputing missing Rn values. Our third purpose, though no less important from the viewpoint of physics, is identifying to which degree physical, in this case environmental variables, are relevant as Rn predictors, or in other words, which predictors explain most of the temporal variability of Rn. We show that variables which contribute most to the Rn series reconstruction, are temperature, relative humidity and day of the year. The first two are physical predictors, while "day of the year" is a statistical proxy or surrogate for missing or unknown predictors. Copyright © 2018 Elsevier B.V. All rights reserved.

Children's Autonomy and Perceived Control in Learning: A Model of Motivation and Achievement in Taiwan.

ERIC Educational Resources Information Center

d'Ailly, Hsiao

2003-01-01

Tests a model of motivation and achievement with data from 50 teachers and 806 Grade 4-6 students in Taiwan. Autonomy as a construct was shown to have ecological validity in Chinese children. The proposed model fit the data well, showing that maternal involvement and autonomy support, as well as teachers' autonomy support, are important for…

Imitation and local enhancement: detrimental effects of consensus definitions on analyses of social learning in animals.

PubMed

Galef, Bennett G

2013-11-01

Development of a widely accepted vocabulary referring to various types of social learning has made important contributions to decades of progress in analyzing the role of socially acquired information in the development of behavioral repertoires. It is argued here that emergence of a consensus vocabulary, while facilitating both communication and research, has also unnecessarily restricted research on social learning. The article has two parts. In the first, I propose that Thorndike's (1898, 1911) definition of imitation as "learning to do an act from seeing it done" has unduly restricted studies of the behavioral processes involved in the propagation of behavior. In part 2, I consider the possibility that success in labeling social learning processes believed to be less cognitively demanding than imitation (e.g. local and stimulus enhancement, social facilitation, etc.) has been mistaken for understanding of those processes, although essentially nothing is known of their stimulus control, development, phylogeny or substrate either behavioral or physiological. Copyright © 2013 Elsevier B.V. All rights reserved.

Deep Learning for Flow Sculpting: Insights into Efficient Learning using Scientific Simulation Data

NASA Astrophysics Data System (ADS)

Stoecklein, Daniel; Lore, Kin Gwn; Davies, Michael; Sarkar, Soumik; Ganapathysubramanian, Baskar

2017-04-01

A new technique for shaping microfluid flow, known as flow sculpting, offers an unprecedented level of passive fluid flow control, with potential breakthrough applications in advancing manufacturing, biology, and chemistry research at the microscale. However, efficiently solving the inverse problem of designing a flow sculpting device for a desired fluid flow shape remains a challenge. Current approaches struggle with the many-to-one design space, requiring substantial user interaction and the necessity of building intuition, all of which are time and resource intensive. Deep learning has emerged as an efficient function approximation technique for high-dimensional spaces, and presents a fast solution to the inverse problem, yet the science of its implementation in similarly defined problems remains largely unexplored. We propose that deep learning methods can completely outpace current approaches for scientific inverse problems while delivering comparable designs. To this end, we show how intelligent sampling of the design space inputs can make deep learning methods more competitive in accuracy, while illustrating their generalization capability to out-of-sample predictions.

A Semisupervised Support Vector Machines Algorithm for BCI Systems

PubMed Central

Qin, Jianzhao; Li, Yuanqing; Sun, Wei

2007-01-01

As an emerging technology, brain-computer interfaces (BCIs) bring us new communication interfaces which translate brain activities into control signals for devices like computers, robots, and so forth. In this study, we propose a semisupervised support vector machine (SVM) algorithm for brain-computer interface (BCI) systems, aiming at reducing the time-consuming training process. In this algorithm, we apply a semisupervised SVM for translating the features extracted from the electrical recordings of brain into control signals. This SVM classifier is built from a small labeled data set and a large unlabeled data set. Meanwhile, to reduce the time for training semisupervised SVM, we propose a batch-mode incremental learning method, which can also be easily applied to the online BCI systems. Additionally, it is suggested in many studies that common spatial pattern (CSP) is very effective in discriminating two different brain states. However, CSP needs a sufficient labeled data set. In order to overcome the drawback of CSP, we suggest a two-stage feature extraction method for the semisupervised learning algorithm. We apply our algorithm to two BCI experimental data sets. The offline data analysis results demonstrate the effectiveness of our algorithm. PMID:18368141

[The personality of obese persons in psychological tests with special consideration on latent obesity].

PubMed

Pudel, V; Metzdorff, M; Oetting, M

1975-01-01

The results of psychological tests of the obese are inconsistent and no characteristic personality structure of the obese can be deduced from them. Investigations in childhood obesity failed to establish a general psychogenetic model of obesity. Yet overweight and ideal weight-subjects differ in spontaneous eating behaviour. Appetite and satiety of obese subjects are controlled by external stimuli to a far greater extent than in nonobese. From a behavioural scientific viewpoint it is proposed that learning experiences during childhood socialisation generate the disposition for obesity which can manifest itself later, after interaction with a special environment. At this stage, however, individual reactions to starting overweight are insolved; this process is strongly influenced by individual personality structures: an inadequate conflict management favours obesity; by cognitive control normal weight can be preserved in spite of the acquired disposition for obesity. Taking these "latently obese" as an example the role of personality structure and wrong eating habits is discussed and related to possible therapeutic strategies. A model of the psychogenetic basis of obesity is proposed. In this model eating-related learning experience is attributed a primary role and individual personality structure a secondary role in the psychogenesis of obesity.

EEG classification for motor imagery and resting state in BCI applications using multi-class Adaboost extreme learning machine

NASA Astrophysics Data System (ADS)

Gao, Lin; Cheng, Wei; Zhang, Jinhua; Wang, Jue

2016-08-01

Brain-computer interface (BCI) systems provide an alternative communication and control approach for people with limited motor function. Therefore, the feature extraction and classification approach should differentiate the relative unusual state of motion intention from a common resting state. In this paper, we sought a novel approach for multi-class classification in BCI applications. We collected electroencephalographic (EEG) signals registered by electrodes placed over the scalp during left hand motor imagery, right hand motor imagery, and resting state for ten healthy human subjects. We proposed using the Kolmogorov complexity (Kc) for feature extraction and a multi-class Adaboost classifier with extreme learning machine as base classifier for classification, in order to classify the three-class EEG samples. An average classification accuracy of 79.5% was obtained for ten subjects, which greatly outperformed commonly used approaches. Thus, it is concluded that the proposed method could improve the performance for classification of motor imagery tasks for multi-class samples. It could be applied in further studies to generate the control commands to initiate the movement of a robotic exoskeleton or orthosis, which finally facilitates the rehabilitation of disabled people.

Low power femtosecond tip-based nanofabrication with advanced control

NASA Astrophysics Data System (ADS)

Liu, Jiangbo; Guo, Zhixiong; Zou, Qingze

2018-02-01

In this paper, we propose an approach to enable the use of low power femtosecond laser in tip-based nanofabrication (TBN) without thermal damage. One major challenge in laser-assisted TBN is in maintaining precision control of the tip-surface positioning throughout the fabrication process. An advanced iterative learning control technique is exploited to overcome this challenge in achieving high-quality patterning of arbitrary shape on a metal surface. The experimental results are analyzed to understand the ablation mechanism involved. Specifically, the near-field radiation enhancement is examined via the surface-enhanced Raman scattering effect, and it was revealed the near-field enhanced plasma-mediated ablation. Moreover, silicon nitride tip is utilized to alleviate the adverse thermal damage. Experiment results including line patterns fabricated under different writing speeds and an "R" pattern are presented. The fabrication quality with regard to the line width, depth, and uniformity is characterized to demonstrate the efficacy of the proposed approach.