Linear System Control Using Stochastic Learning Automata

NASA Technical Reports Server (NTRS)

Ziyad, Nigel; Cox, E. Lucien; Chouikha, Mohamed F.

1998-01-01

This paper explains the use of a Stochastic Learning Automata (SLA) to control switching between three systems to produce the desired output response. The SLA learns the optimal choice of the damping ratio for each system to achieve a desired result. We show that the SLA can learn these states for the control of an unknown system with the proper choice of the error criteria. The results of using a single automaton are compared to using multiple automata.

Intelligent control based on fuzzy logic and neural net theory

NASA Technical Reports Server (NTRS)

Lee, Chuen-Chien

1991-01-01

In the conception and design of intelligent systems, one promising direction involves the use of fuzzy logic and neural network theory to enhance such systems' capability to learn from experience and adapt to changes in an environment of uncertainty and imprecision. Here, an intelligent control scheme is explored by integrating these multidisciplinary techniques. A self-learning system is proposed as an intelligent controller for dynamical processes, employing a control policy which evolves and improves automatically. One key component of the intelligent system is a fuzzy logic-based system which emulates human decision making behavior. It is shown that the system can solve a fairly difficult control learning problem. Simulation results demonstrate that improved learning performance can be achieved in relation to previously described systems employing bang-bang control. The proposed system is relatively insensitive to variations in the parameters of the system environment.

Dynamic learning from adaptive neural network control of a class of nonaffine nonlinear systems.

PubMed

Dai, Shi-Lu; Wang, Cong; Wang, Min

2014-01-01

This paper studies the problem of learning from adaptive neural network (NN) control of a class of nonaffine nonlinear systems in uncertain dynamic environments. In the control design process, a stable adaptive NN tracking control design technique is proposed for the nonaffine nonlinear systems with a mild assumption by combining a filtered tracking error with the implicit function theorem, input-to-state stability, and the small-gain theorem. The proposed stable control design technique not only overcomes the difficulty in controlling nonaffine nonlinear systems but also relaxes constraint conditions of the considered systems. In the learning process, the partial persistent excitation (PE) condition of radial basis function NNs is satisfied during tracking control to a recurrent reference trajectory. Under the PE condition and an appropriate state transformation, the proposed adaptive NN control is shown to be capable of acquiring knowledge on the implicit desired control input dynamics in the stable control process and of storing the learned knowledge in memory. Subsequently, an NN learning control design technique that effectively exploits the learned knowledge without re-adapting to the controller parameters is proposed to achieve closed-loop stability and improved control performance. Simulation studies are performed to demonstrate the effectiveness of the proposed design techniques.

Performance improvement in remote manipulation with time delay by means of a learning system.

NASA Technical Reports Server (NTRS)

Freedy, A.; Weltman, G.

1973-01-01

A teleoperating system is presented that involves shared control between a human operator and a general-purpose computer-based learning machine. This setup features a trainable control network termed the autonomous control subsystem (ACS) which is able to observe the operator's control actions, learn the task at hand, and take appropriate control actions. A working ACS system is described that has been put in operation for the purpose of exploring the uses of a remote intelligence of this type. The expansion of the present system into a multifunctional learning machine capable of a greater degree of autonomy is also discussed.

Neurofeedback Control of the Human GABAergic System Using Non-invasive Brain Stimulation.

PubMed

Koganemaru, Satoko; Mikami, Yusuke; Maezawa, Hitoshi; Ikeda, Satoshi; Ikoma, Katsunori; Mima, Tatsuya

2018-06-01

Neurofeedback has been a powerful method for self-regulating brain activities to elicit potential ability of human mind. GABA is a major inhibitory neurotransmitter in the central nervous system. Transcranial magnetic stimulation (TMS) is a tool that can evaluate the GABAergic system within the primary motor cortex (M1) using paired-pulse stimuli, short intracortical inhibition (SICI). Herein we investigated whether neurofeedback learning using SICI enabled us to control the GABAergic system within the M1 area. Forty-five healthy subjects were randomly divided into two groups: those receiving SICI neurofeedback learning or those receiving no neurofeedback (control) learning. During both learning periods, subjects made attempts to change the size of a circle, which was altered according to the degree of SICI in the SICI neurofeedback learning group, and which was altered independent of the degree of SICI in the control learning group. Results demonstrated that the SICI neurofeedback learning group showed a significant enhancement in SICI. Moreover, this group showed a significant reduction in choice reaction time compared to the control group. Our findings indicate that humans can intrinsically control the intracortical GABAergic system within M1 and can thus improve motor behaviors by SICI neurofeedback learning. SICI neurofeedback learning is a novel and promising approach to control our neural system and potentially represents a new therapy for patients with abnormal motor symptoms caused by CNS disorders. Copyright © 2018 IBRO. Published by Elsevier Ltd. All rights reserved.

Robust design of feedback feed-forward iterative learning control based on 2D system theory for linear uncertain systems

NASA Astrophysics Data System (ADS)

Li, Zhifu; Hu, Yueming; Li, Di

2016-08-01

For a class of linear discrete-time uncertain systems, a feedback feed-forward iterative learning control (ILC) scheme is proposed, which is comprised of an iterative learning controller and two current iteration feedback controllers. The iterative learning controller is used to improve the performance along the iteration direction and the feedback controllers are used to improve the performance along the time direction. First of all, the uncertain feedback feed-forward ILC system is presented by an uncertain two-dimensional Roesser model system. Then, two robust control schemes are proposed. One can ensure that the feedback feed-forward ILC system is bounded-input bounded-output stable along time direction, and the other can ensure that the feedback feed-forward ILC system is asymptotically stable along time direction. Both schemes can guarantee the system is robust monotonically convergent along the iteration direction. Third, the robust convergent sufficient conditions are given, which contains a linear matrix inequality (LMI). Moreover, the LMI can be used to determine the gain matrix of the feedback feed-forward iterative learning controller. Finally, the simulation results are presented to demonstrate the effectiveness of the proposed schemes.

Gaussian Processes for Data-Efficient Learning in Robotics and Control.

PubMed

Deisenroth, Marc Peter; Fox, Dieter; Rasmussen, Carl Edward

2015-02-01

Autonomous learning has been a promising direction in control and robotics for more than a decade since data-driven learning allows to reduce the amount of engineering knowledge, which is otherwise required. However, autonomous reinforcement learning (RL) approaches typically require many interactions with the system to learn controllers, which is a practical limitation in real systems, such as robots, where many interactions can be impractical and time consuming. To address this problem, current learning approaches typically require task-specific knowledge in form of expert demonstrations, realistic simulators, pre-shaped policies, or specific knowledge about the underlying dynamics. In this paper, we follow a different approach and speed up learning by extracting more information from data. In particular, we learn a probabilistic, non-parametric Gaussian process transition model of the system. By explicitly incorporating model uncertainty into long-term planning and controller learning our approach reduces the effects of model errors, a key problem in model-based learning. Compared to state-of-the art RL our model-based policy search method achieves an unprecedented speed of learning. We demonstrate its applicability to autonomous learning in real robot and control tasks.

Learning and tuning fuzzy logic controllers through reinforcements.

PubMed

Berenji, H R; Khedkar, P

1992-01-01

A method for learning and tuning a fuzzy logic controller based on reinforcements from a dynamic system is presented. It is shown that: the generalized approximate-reasoning-based intelligent control (GARIC) architecture learns and tunes a fuzzy logic controller even when only weak reinforcement, such as a binary failure signal, is available; introduces a new conjunction operator in computing the rule strengths of fuzzy control rules; introduces a new localized mean of maximum (LMOM) method in combining the conclusions of several firing control rules; and learns to produce real-valued control actions. Learning is achieved by integrating fuzzy inference into a feedforward network, which can then adaptively improve performance by using gradient descent methods. The GARIC architecture is applied to a cart-pole balancing system and demonstrates significant improvements in terms of the speed of learning and robustness to changes in the dynamic system's parameters over previous schemes for cart-pole balancing.

Adaptive/learning control of large space structures - System identification techniques. [for multi-configuration flexible spacecraft

NASA Technical Reports Server (NTRS)

Thau, F. E.; Montgomery, R. C.

1980-01-01

Techniques developed for the control of aircraft under changing operating conditions are used to develop a learning control system structure for a multi-configuration, flexible space vehicle. A configuration identification subsystem that is to be used with a learning algorithm and a memory and control process subsystem is developed. Adaptive gain adjustments can be achieved by this learning approach without prestoring of large blocks of parameter data and without dither signal inputs which will be suppressed during operations for which they are not compatible. The Space Shuttle Solar Electric Propulsion (SEP) experiment is used as a sample problem for the testing of adaptive/learning control system algorithms.

Research on intelligent algorithm of electro - hydraulic servo control system

NASA Astrophysics Data System (ADS)

Wang, Yannian; Zhao, Yuhui; Liu, Chengtao

2017-09-01

In order to adapt the nonlinear characteristics of the electro-hydraulic servo control system and the influence of complex interference in the industrial field, using a fuzzy PID switching learning algorithm is proposed and a fuzzy PID switching learning controller is designed and applied in the electro-hydraulic servo controller. The designed controller not only combines the advantages of the fuzzy control and PID control, but also introduces the learning algorithm into the switching function, which makes the learning of the three parameters in the switching function can avoid the instability of the system during the switching between the fuzzy control and PID control algorithms. It also makes the switch between these two control algorithm more smoother than that of the conventional fuzzy PID.

Learning from adaptive neural dynamic surface control of strict-feedback systems.

PubMed

Wang, Min; Wang, Cong

2015-06-01

Learning plays an essential role in autonomous control systems. However, how to achieve learning in the nonstationary environment for nonlinear systems is a challenging problem. In this paper, we present learning method for a class of n th-order strict-feedback systems by adaptive dynamic surface control (DSC) technology, which achieves the human-like ability of learning by doing and doing with learned knowledge. To achieve the learning, this paper first proposes stable adaptive DSC with auxiliary first-order filters, which ensures the boundedness of all the signals in the closed-loop system and the convergence of tracking errors in a finite time. With the help of DSC, the derivative of the filter output variable is used as the neural network (NN) input instead of traditional intermediate variables. As a result, the proposed adaptive DSC method reduces greatly the dimension of NN inputs, especially for high-order systems. After the stable DSC design, we decompose the stable closed-loop system into a series of linear time-varying perturbed subsystems. Using a recursive design, the recurrent property of NN input variables is easily verified since the complexity is overcome using DSC. Subsequently, the partial persistent excitation condition of the radial basis function NN is satisfied. By combining a state transformation, accurate approximations of the closed-loop system dynamics are recursively achieved in a local region along recurrent orbits. Then, the learning control method using the learned knowledge is proposed to achieve the closed-loop stability and the improved control performance. Simulation studies are performed to demonstrate the proposed scheme can not only reuse the learned knowledge to achieve the better control performance with the faster tracking convergence rate and the smaller tracking error but also greatly alleviate the computational burden because of reducing the number and complexity of NN input variables.

Learning fuzzy logic control system

NASA Technical Reports Server (NTRS)

Lung, Leung Kam

1994-01-01

The performance of the Learning Fuzzy Logic Control System (LFLCS), developed in this thesis, has been evaluated. The Learning Fuzzy Logic Controller (LFLC) learns to control the motor by learning the set of teaching values that are generated by a classical PI controller. It is assumed that the classical PI controller is tuned to minimize the error of a position control system of the D.C. motor. The Learning Fuzzy Logic Controller developed in this thesis is a multi-input single-output network. Training of the Learning Fuzzy Logic Controller is implemented off-line. Upon completion of the training process (using Supervised Learning, and Unsupervised Learning), the LFLC replaces the classical PI controller. In this thesis, a closed loop position control system of a D.C. motor using the LFLC is implemented. The primary focus is on the learning capabilities of the Learning Fuzzy Logic Controller. The learning includes symbolic representation of the Input Linguistic Nodes set and Output Linguistic Notes set. In addition, we investigate the knowledge-based representation for the network. As part of the design process, we implement a digital computer simulation of the LFLCS. The computer simulation program is written in 'C' computer language, and it is implemented in DOS platform. The LFLCS, designed in this thesis, has been developed on a IBM compatible 486-DX2 66 computer. First, the performance of the Learning Fuzzy Logic Controller is evaluated by comparing the angular shaft position of the D.C. motor controlled by a conventional PI controller and that controlled by the LFLC. Second, the symbolic representation of the LFLC and the knowledge-based representation for the network are investigated by observing the parameters of the Fuzzy Logic membership functions and the links at each layer of the LFLC. While there are some limitations of application with this approach, the result of the simulation shows that the LFLC is able to control the angular shaft position of the D.C. motor. Furthermore, the LFLC has better performance in rise time, settling time and steady state error than to the conventional PI controller. This abstract accurately represents the content of the candidate's thesis. I recommend its publication.

Exploring Effects of Multi-Touch Tabletop on Collaborative Fraction Learning and the Relationship of Learning Behavior and Interaction with Learning Achievement

ERIC Educational Resources Information Center

Hwang, Wu-Yuin; Shadiev, Rustam; Tseng, Chi-Wei; Huang, Yueh-Min

2015-01-01

This study designed a learning system to facilitate elementary school students' fraction learning. An experiment was carried out to investigate how the system, which runs on multi-touch tabletop versus tablet PC, affects fraction learning. Two groups, a control and experimental, were assigned. Control students have learned fraction by using tablet…

ISO learning approximates a solution to the inverse-controller problem in an unsupervised behavioral paradigm.

PubMed

Porr, Bernd; von Ferber, Christian; Wörgötter, Florentin

2003-04-01

In "Isotropic Sequence Order Learning" (pp. 831-864 in this issue), we introduced a novel algorithm for temporal sequence learning (ISO learning). Here, we embed this algorithm into a formal nonevaluating (teacher free) environment, which establishes a sensor-motor feedback. The system is initially guided by a fixed reflex reaction, which has the objective disadvantage that it can react only after a disturbance has occurred. ISO learning eliminates this disadvantage by replacing the reflex-loop reactions with earlier anticipatory actions. In this article, we analytically demonstrate that this process can be understood in terms of control theory, showing that the system learns the inverse controller of its own reflex. Thereby, this system is able to learn a simple form of feedforward motor control.

Amplifying human ability through autonomics and machine learning in IMPACT

NASA Astrophysics Data System (ADS)

Dzieciuch, Iryna; Reeder, John; Gutzwiller, Robert; Gustafson, Eric; Coronado, Braulio; Martinez, Luis; Croft, Bryan; Lange, Douglas S.

2017-05-01

Amplifying human ability for controlling complex environments featuring autonomous units can be aided by learned models of human and system performance. In developing a command and control system that allows a small number of people to control a large number of autonomous teams, we employ an autonomics framework to manage the networks that represent mission plans and the networks that are composed of human controllers and their autonomous assistants. Machine learning allows us to build models of human and system performance useful for monitoring plans and managing human attention and task loads. Machine learning also aids in the development of tactics that human supervisors can successfully monitor through the command and control system.

A Hierarchical Learning Control Framework for an Aerial Manipulation System

NASA Astrophysics Data System (ADS)

Ma, Le; Chi, yanxun; Li, Jiapeng; Li, Zhongsheng; Ding, Yalei; Liu, Lixing

2017-07-01

A hierarchical learning control framework for an aerial manipulation system is proposed. Firstly, the mechanical design of aerial manipulation system is introduced and analyzed, and the kinematics and the dynamics based on Newton-Euler equation are modeled. Secondly, the framework of hierarchical learning for this system is presented, in which flight platform and manipulator are controlled by different controller respectively. The RBF (Radial Basis Function) neural networks are employed to estimate parameters and control. The Simulation and experiment demonstrate that the methods proposed effective and advanced.

Developing Learning Tool of Control System Engineering Using Matrix Laboratory Software Oriented on Industrial Needs

NASA Astrophysics Data System (ADS)

Isnur Haryudo, Subuh; Imam Agung, Achmad; Firmansyah, Rifqi

2018-04-01

The purpose of this research is to develop learning media of control technique using Matrix Laboratory software with industry requirement approach. Learning media serves as a tool for creating a better and effective teaching and learning situation because it can accelerate the learning process in order to enhance the quality of learning. Control Techniques using Matrix Laboratory software can enlarge the interest and attention of students, with real experience and can grow independent attitude. This research design refers to the use of research and development (R & D) methods that have been modified by multi-disciplinary team-based researchers. This research used Computer based learning method consisting of computer and Matrix Laboratory software which was integrated with props. Matrix Laboratory has the ability to visualize the theory and analysis of the Control System which is an integration of computing, visualization and programming which is easy to use. The result of this instructional media development is to use mathematical equations using Matrix Laboratory software on control system application with DC motor plant and PID (Proportional-Integral-Derivative). Considering that manufacturing in the field of Distributed Control systems (DCSs), Programmable Controllers (PLCs), and Microcontrollers (MCUs) use PID systems in production processes are widely used in industry.

Optimal and Autonomous Control Using Reinforcement Learning: A Survey.

PubMed

Kiumarsi, Bahare; Vamvoudakis, Kyriakos G; Modares, Hamidreza; Lewis, Frank L

2018-06-01

This paper reviews the current state of the art on reinforcement learning (RL)-based feedback control solutions to optimal regulation and tracking of single and multiagent systems. Existing RL solutions to both optimal and control problems, as well as graphical games, will be reviewed. RL methods learn the solution to optimal control and game problems online and using measured data along the system trajectories. We discuss Q-learning and the integral RL algorithm as core algorithms for discrete-time (DT) and continuous-time (CT) systems, respectively. Moreover, we discuss a new direction of off-policy RL for both CT and DT systems. Finally, we review several applications.

Learning and tuning fuzzy logic controllers through reinforcements

NASA Technical Reports Server (NTRS)

Berenji, Hamid R.; Khedkar, Pratap

1992-01-01

A new method for learning and tuning a fuzzy logic controller based on reinforcements from a dynamic system is presented. In particular, our Generalized Approximate Reasoning-based Intelligent Control (GARIC) architecture: (1) learns and tunes a fuzzy logic controller even when only weak reinforcements, such as a binary failure signal, is available; (2) introduces a new conjunction operator in computing the rule strengths of fuzzy control rules; (3) introduces a new localized mean of maximum (LMOM) method in combining the conclusions of several firing control rules; and (4) learns to produce real-valued control actions. Learning is achieved by integrating fuzzy inference into a feedforward network, which can then adaptively improve performance by using gradient descent methods. We extend the AHC algorithm of Barto, Sutton, and Anderson to include the prior control knowledge of human operators. The GARIC architecture is applied to a cart-pole balancing system and has demonstrated significant improvements in terms of the speed of learning and robustness to changes in the dynamic system's parameters over previous schemes for cart-pole balancing.

An adaptive learning control system for aircraft

NASA Technical Reports Server (NTRS)

Mekel, R.; Nachmias, S.

1978-01-01

A learning control system and its utilization as a flight control system for F-8 Digital Fly-By-Wire (DFBW) research aircraft is studied. The system has the ability to adjust a gain schedule to account for changing plant characteristics and to improve its performance and the plant's performance in the course of its own operation. Three subsystems are detailed: (1) the information acquisition subsystem which identifies the plant's parameters at a given operating condition; (2) the learning algorithm subsystem which relates the identified parameters to predetermined analytical expressions describing the behavior of the parameters over a range of operating conditions; and (3) the memory and control process subsystem which consists of the collection of updated coefficients (memory) and the derived control laws. Simulation experiments indicate that the learning control system is effective in compensating for parameter variations caused by changes in flight conditions.

Direct heuristic dynamic programming for damping oscillations in a large power system.

PubMed

Lu, Chao; Si, Jennie; Xie, Xiaorong

2008-08-01

This paper applies a neural-network-based approximate dynamic programming method, namely, the direct heuristic dynamic programming (direct HDP), to a large power system stability control problem. The direct HDP is a learning- and approximation-based approach to addressing nonlinear coordinated control under uncertainty. One of the major design parameters, the controller learning objective function, is formulated to directly account for network-wide low-frequency oscillation with the presence of nonlinearity, uncertainty, and coupling effect among system components. Results include a novel learning control structure based on the direct HDP with applications to two power system problems. The first case involves static var compensator supplementary damping control, which is used to provide a comprehensive evaluation of the learning control performance. The second case aims at addressing a difficult complex system challenge by providing a new solution to a large interconnected power network oscillation damping control problem that frequently occurs in the China Southern Power Grid.

Fault-tolerant nonlinear adaptive flight control using sliding mode online learning.

PubMed

Krüger, Thomas; Schnetter, Philipp; Placzek, Robin; Vörsmann, Peter

2012-08-01

An expanded nonlinear model inversion flight control strategy using sliding mode online learning for neural networks is presented. The proposed control strategy is implemented for a small unmanned aircraft system (UAS). This class of aircraft is very susceptible towards nonlinearities like atmospheric turbulence, model uncertainties and of course system failures. Therefore, these systems mark a sensible testbed to evaluate fault-tolerant, adaptive flight control strategies. Within this work the concept of feedback linearization is combined with feed forward neural networks to compensate for inversion errors and other nonlinear effects. Backpropagation-based adaption laws of the network weights are used for online training. Within these adaption laws the standard gradient descent backpropagation algorithm is augmented with the concept of sliding mode control (SMC). Implemented as a learning algorithm, this nonlinear control strategy treats the neural network as a controlled system and allows a stable, dynamic calculation of the learning rates. While considering the system's stability, this robust online learning method therefore offers a higher speed of convergence, especially in the presence of external disturbances. The SMC-based flight controller is tested and compared with the standard gradient descent backpropagation algorithm in the presence of system failures. Copyright © 2012 Elsevier Ltd. All rights reserved.

GA-based fuzzy reinforcement learning for control of a magnetic bearing system.

PubMed

Lin, C T; Jou, C P

2000-01-01

This paper proposes a TD (temporal difference) and GA (genetic algorithm)-based reinforcement (TDGAR) learning method and applies it to the control of a real magnetic bearing system. The TDGAR learning scheme is a new hybrid GA, which integrates the TD prediction method and the GA to perform the reinforcement learning task. The TDGAR learning system is composed of two integrated feedforward networks. One neural network acts as a critic network to guide the learning of the other network (the action network) which determines the outputs (actions) of the TDGAR learning system. The action network can be a normal neural network or a neural fuzzy network. Using the TD prediction method, the critic network can predict the external reinforcement signal and provide a more informative internal reinforcement signal to the action network. The action network uses the GA to adapt itself according to the internal reinforcement signal. The key concept of the TDGAR learning scheme is to formulate the internal reinforcement signal as the fitness function for the GA such that the GA can evaluate the candidate solutions (chromosomes) regularly, even during periods without external feedback from the environment. This enables the GA to proceed to new generations regularly without waiting for the arrival of the external reinforcement signal. This can usually accelerate the GA learning since a reinforcement signal may only be available at a time long after a sequence of actions has occurred in the reinforcement learning problem. The proposed TDGAR learning system has been used to control an active magnetic bearing (AMB) system in practice. A systematic design procedure is developed to achieve successful integration of all the subsystems including magnetic suspension, mechanical structure, and controller training. The results show that the TDGAR learning scheme can successfully find a neural controller or a neural fuzzy controller for a self-designed magnetic bearing system.

Closed-loop and robust control of quantum systems.

PubMed

Chen, Chunlin; Wang, Lin-Cheng; Wang, Yuanlong

2013-01-01

For most practical quantum control systems, it is important and difficult to attain robustness and reliability due to unavoidable uncertainties in the system dynamics or models. Three kinds of typical approaches (e.g., closed-loop learning control, feedback control, and robust control) have been proved to be effective to solve these problems. This work presents a self-contained survey on the closed-loop and robust control of quantum systems, as well as a brief introduction to a selection of basic theories and methods in this research area, to provide interested readers with a general idea for further studies. In the area of closed-loop learning control of quantum systems, we survey and introduce such learning control methods as gradient-based methods, genetic algorithms (GA), and reinforcement learning (RL) methods from a unified point of view of exploring the quantum control landscapes. For the feedback control approach, the paper surveys three control strategies including Lyapunov control, measurement-based control, and coherent-feedback control. Then such topics in the field of quantum robust control as H(∞) control, sliding mode control, quantum risk-sensitive control, and quantum ensemble control are reviewed. The paper concludes with a perspective of future research directions that are likely to attract more attention.

Iterative learning-based decentralized adaptive tracker for large-scale systems: a digital redesign approach.

PubMed

Tsai, Jason Sheng-Hong; Du, Yan-Yi; Huang, Pei-Hsiang; Guo, Shu-Mei; Shieh, Leang-San; Chen, Yuhua

2011-07-01

In this paper, a digital redesign methodology of the iterative learning-based decentralized adaptive tracker is proposed to improve the dynamic performance of sampled-data linear large-scale control systems consisting of N interconnected multi-input multi-output subsystems, so that the system output will follow any trajectory which may not be presented by the analytic reference model initially. To overcome the interference of each sub-system and simplify the controller design, the proposed model reference decentralized adaptive control scheme constructs a decoupled well-designed reference model first. Then, according to the well-designed model, this paper develops a digital decentralized adaptive tracker based on the optimal analog control and prediction-based digital redesign technique for the sampled-data large-scale coupling system. In order to enhance the tracking performance of the digital tracker at specified sampling instants, we apply the iterative learning control (ILC) to train the control input via continual learning. As a result, the proposed iterative learning-based decentralized adaptive tracker not only has robust closed-loop decoupled property but also possesses good tracking performance at both transient and steady state. Besides, evolutionary programming is applied to search for a good learning gain to speed up the learning process of ILC. Copyright © 2011 ISA. Published by Elsevier Ltd. All rights reserved.

Intelligent fault-tolerant controllers

NASA Technical Reports Server (NTRS)

Huang, Chien Y.

1987-01-01

A system with fault tolerant controls is one that can detect, isolate, and estimate failures and perform necessary control reconfiguration based on this new information. Artificial intelligence (AI) is concerned with semantic processing, and it has evolved to include the topics of expert systems and machine learning. This research represents an attempt to apply AI to fault tolerant controls, hence, the name intelligent fault tolerant control (IFTC). A generic solution to the problem is sought, providing a system based on logic in addition to analytical tools, and offering machine learning capabilities. The advantages are that redundant system specific algorithms are no longer needed, that reasonableness is used to quickly choose the correct control strategy, and that the system can adapt to new situations by learning about its effects on system dynamics.

How does a specific learning and memory system in the mammalian brain gain control of behavior?

PubMed

McDonald, Robert J; Hong, Nancy S

2013-11-01

This review addresses a fundamental, yet poorly understood set of issues in systems neuroscience. The issues revolve around conceptualizations of the organization of learning and memory in the mammalian brain. One intriguing, and somewhat popular, conceptualization is the idea that there are multiple learning and memory systems in the mammalian brain and they interact in different ways to influence and/or control behavior. This approach has generated interesting empirical and theoretical work supporting this view. One issue that needs to be addressed is how these systems influence or gain control of voluntary behavior. To address this issue, we clearly specify what we mean by a learning and memory system. We then review two types of processes that might influence which memory system gains control of behavior. One set of processes are external factors that can affect which system controls behavior in a given situation including task parameters like the kind of information available to the subject, types of training experience, and amount of training. The second set of processes are brain mechanisms that might influence what memory system controls behavior in a given situation including executive functions mediated by the prefrontal cortex; switching mechanisms mediated by ascending neurotransmitter systems, the unique role of the hippocampus during learning. The issue of trait differences in control of different learning and memory systems will also be considered in which trait differences in learning and memory function are thought to potentially emerge from differences in level of prefrontal influence, differences in plasticity processes, differences in ascending neurotransmitter control, differential access to effector systems like motivational and motor systems. Finally, we present scenarios in which different mechanisms might interact. This review was conceived to become a jumping off point for new work directed at understanding these issues. The outcome of this work, in combination with other approaches, might improve understanding of the mechanisms of volition in human and non-human animals. Copyright © 2013 Wiley Periodicals, Inc.

Development of Advanced Verification and Validation Procedures and Tools for the Certification of Learning Systems in Aerospace Applications

NASA Technical Reports Server (NTRS)

Jacklin, Stephen; Schumann, Johann; Gupta, Pramod; Richard, Michael; Guenther, Kurt; Soares, Fola

2005-01-01

Adaptive control technologies that incorporate learning algorithms have been proposed to enable automatic flight control and vehicle recovery, autonomous flight, and to maintain vehicle performance in the face of unknown, changing, or poorly defined operating environments. In order for adaptive control systems to be used in safety-critical aerospace applications, they must be proven to be highly safe and reliable. Rigorous methods for adaptive software verification and validation must be developed to ensure that control system software failures will not occur. Of central importance in this regard is the need to establish reliable methods that guarantee convergent learning, rapid convergence (learning) rate, and algorithm stability. This paper presents the major problems of adaptive control systems that use learning to improve performance. The paper then presents the major procedures and tools presently developed or currently being developed to enable the verification, validation, and ultimate certification of these adaptive control systems. These technologies include the application of automated program analysis methods, techniques to improve the learning process, analytical methods to verify stability, methods to automatically synthesize code, simulation and test methods, and tools to provide on-line software assurance.

A learning flight control system for the F8-DFBW aircraft. [Digital Fly-By-Wire

NASA Technical Reports Server (NTRS)

Montgomery, R. C.; Mekel, R.; Nachmias, S.

1978-01-01

This report contains a complete description of a learning control system designed for the F8-DFBW aircraft. The system is parameter-adaptive with the additional feature that it 'learns' the variation of the control system gains needed over the flight envelope. It, thus, generates and modifies its gain schedule when suitable data are available. The report emphasizes the novel learning features of the system: the forms of representation of the flight envelope and the process by which identified parameters are used to modify the gain schedule. It contains data taken during piloted real-time 6 degree-of-freedom simulations that were used to develop and evaluate the system.

A learning controller for nonrepetitive robotic operation

NASA Technical Reports Server (NTRS)

Miller, W. T., III

1987-01-01

A practical learning control system is described which is applicable to complex robotic and telerobotic systems involving multiple feedback sensors and multiple command variables. In the controller, the learning algorithm is used to learn to reproduce the nonlinear relationship between the sensor outputs and the system command variables over particular regions of the system state space, rather than learning the actuator commands required to perform a specific task. The learned information is used to predict the command signals required to produce desired changes in the sensor outputs. The desired sensor output changes may result from automatic trajectory planning or may be derived from interactive input from a human operator. The learning controller requires no a priori knowledge of the relationships between the sensor outputs and the command variables. The algorithm is well suited for real time implementation, requiring only fixed point addition and logical operations. The results of learning experiments using a General Electric P-5 manipulator interfaced to a VAX-11/730 computer are presented. These experiments involved interactive operator control, via joysticks, of the position and orientation of an object in the field of view of a video camera mounted on the end of the robot arm.

Techniques for improving transients in learning control systems

NASA Technical Reports Server (NTRS)

Chang, C.-K.; Longman, Richard W.; Phan, Minh

1992-01-01

A discrete modern control formulation is used to study the nature of the transient behavior of the learning process during repetitions. Several alternative learning control schemes are developed to improve the transient performance. These include a new method using an alternating sign on the learning gain, which is very effective in limiting peak transients and also very useful in multiple-input, multiple-output systems. Other methods include learning at an increasing number of points progressing with time, or an increasing number of points of increasing density.

Reinforcement learning for partially observable dynamic processes: adaptive dynamic programming using measured output data.

PubMed

Lewis, F L; Vamvoudakis, Kyriakos G

2011-02-01

Approximate dynamic programming (ADP) is a class of reinforcement learning methods that have shown their importance in a variety of applications, including feedback control of dynamical systems. ADP generally requires full information about the system internal states, which is usually not available in practical situations. In this paper, we show how to implement ADP methods using only measured input/output data from the system. Linear dynamical systems with deterministic behavior are considered herein, which are systems of great interest in the control system community. In control system theory, these types of methods are referred to as output feedback (OPFB). The stochastic equivalent of the systems dealt with in this paper is a class of partially observable Markov decision processes. We develop both policy iteration and value iteration algorithms that converge to an optimal controller that requires only OPFB. It is shown that, similar to Q -learning, the new methods have the important advantage that knowledge of the system dynamics is not needed for the implementation of these learning algorithms or for the OPFB control. Only the order of the system, as well as an upper bound on its "observability index," must be known. The learned OPFB controller is in the form of a polynomial autoregressive moving-average controller that has equivalent performance with the optimal state variable feedback gain.

Artificial neural networks and approximate reasoning for intelligent control in space

NASA Technical Reports Server (NTRS)

Berenji, Hamid R.

1991-01-01

A method is introduced for learning to refine the control rules of approximate reasoning-based controllers. A reinforcement-learning technique is used in conjunction with a multi-layer neural network model of an approximate reasoning-based controller. The model learns by updating its prediction of the physical system's behavior. The model can use the control knowledge of an experienced operator and fine-tune it through the process of learning. Some of the space domains suitable for applications of the model such as rendezvous and docking, camera tracking, and tethered systems control are discussed.

Indirect decentralized repetitive control

NASA Technical Reports Server (NTRS)

Lee, Soo Cheol; Longman, Richard W.

1993-01-01

Learning control refers to controllers that learn to improve their performance at executing a given task, based on experience performing this specific task. In a previous work, the authors presented a theory of indirect decentralized learning control based on use of indirect adaptive control concepts employing simultaneous identification and control. This paper extends these results to apply to the indirect repetitive control problem in which a periodic (i.e., repetitive) command is given to a control system. Decentralized indirect repetitive control algorithms are presented that have guaranteed convergence to zero tracking error under very general conditions. The original motivation of the repetitive control and learning control fields was learning in robots doing repetitive tasks such as on an assembly line. This paper starts with decentralized discrete time systems, and progresses to the robot application, modeling the robot as a time varying linear system in the neighborhood of the desired trajectory. Decentralized repetitive control is natural for this application because the feedback control for link rotations is normally implemented in a decentralized manner, treating each link as if it is independent of the other links.

Effectiveness of Adaptive Assessment versus Learner Control in a Multimedia Learning System

ERIC Educational Resources Information Center

Chen, Ching-Huei; Chang, Shu-Wei

2015-01-01

The purpose of this study was to explore the effectiveness of adaptive assessment versus learner control in a multimedia learning system designed to help secondary students learn science. Unlike other systems, this paper presents a workflow of adaptive assessment following instructional materials that better align with learners' cognitive…

Learning and tuning fuzzy logic controllers through reinforcements

NASA Technical Reports Server (NTRS)

Berenji, Hamid R.; Khedkar, Pratap

1992-01-01

This paper presents a new method for learning and tuning a fuzzy logic controller based on reinforcements from a dynamic system. In particular, our generalized approximate reasoning-based intelligent control (GARIC) architecture (1) learns and tunes a fuzzy logic controller even when only weak reinforcement, such as a binary failure signal, is available; (2) introduces a new conjunction operator in computing the rule strengths of fuzzy control rules; (3) introduces a new localized mean of maximum (LMOM) method in combining the conclusions of several firing control rules; and (4) learns to produce real-valued control actions. Learning is achieved by integrating fuzzy inference into a feedforward neural network, which can then adaptively improve performance by using gradient descent methods. We extend the AHC algorithm of Barto et al. (1983) to include the prior control knowledge of human operators. The GARIC architecture is applied to a cart-pole balancing system and demonstrates significant improvements in terms of the speed of learning and robustness to changes in the dynamic system's parameters over previous schemes for cart-pole balancing.

The curse of planning: dissecting multiple reinforcement-learning systems by taxing the central executive.

PubMed

Otto, A Ross; Gershman, Samuel J; Markman, Arthur B; Daw, Nathaniel D

2013-05-01

A number of accounts of human and animal behavior posit the operation of parallel and competing valuation systems in the control of choice behavior. In these accounts, a flexible but computationally expensive model-based reinforcement-learning system has been contrasted with a less flexible but more efficient model-free reinforcement-learning system. The factors governing which system controls behavior-and under what circumstances-are still unclear. Following the hypothesis that model-based reinforcement learning requires cognitive resources, we demonstrated that having human decision makers perform a demanding secondary task engenders increased reliance on a model-free reinforcement-learning strategy. Further, we showed that, across trials, people negotiate the trade-off between the two systems dynamically as a function of concurrent executive-function demands, and people's choice latencies reflect the computational expenses of the strategy they employ. These results demonstrate that competition between multiple learning systems can be controlled on a trial-by-trial basis by modulating the availability of cognitive resources.

Evolving fuzzy rules in a learning classifier system

NASA Technical Reports Server (NTRS)

Valenzuela-Rendon, Manuel

1993-01-01

The fuzzy classifier system (FCS) combines the ideas of fuzzy logic controllers (FLC's) and learning classifier systems (LCS's). It brings together the expressive powers of fuzzy logic as it has been applied in fuzzy controllers to express relations between continuous variables, and the ability of LCS's to evolve co-adapted sets of rules. The goal of the FCS is to develop a rule-based system capable of learning in a reinforcement regime, and that can potentially be used for process control.

A statistical learning strategy for closed-loop control of fluid flows

NASA Astrophysics Data System (ADS)

Guéniat, Florimond; Mathelin, Lionel; Hussaini, M. Yousuff

2016-12-01

This work discusses a closed-loop control strategy for complex systems utilizing scarce and streaming data. A discrete embedding space is first built using hash functions applied to the sensor measurements from which a Markov process model is derived, approximating the complex system's dynamics. A control strategy is then learned using reinforcement learning once rewards relevant with respect to the control objective are identified. This method is designed for experimental configurations, requiring no computations nor prior knowledge of the system, and enjoys intrinsic robustness. It is illustrated on two systems: the control of the transitions of a Lorenz'63 dynamical system, and the control of the drag of a cylinder flow. The method is shown to perform well.

Design of robust iterative learning control schemes for systems with polytopic uncertainties and sector-bounded nonlinearities

NASA Astrophysics Data System (ADS)

Boski, Marcin; Paszke, Wojciech

2017-01-01

This paper deals with designing of iterative learning control schemes for uncertain systems with static nonlinearities. More specifically, the nonlinear part is supposed to be sector bounded and system matrices are assumed to range in the polytope of matrices. For systems with such nonlinearities and uncertainties the repetitive process setting is exploited to develop a linear matrix inequality based conditions for computing the feedback and feedforward (learning) controllers. These controllers guarantee acceptable dynamics along the trials and ensure convergence of the trial-to-trial error dynamics, respectively. Numerical examples illustrate the theoretical results and confirm effectiveness of the designed control scheme.

Closed-Loop and Robust Control of Quantum Systems

PubMed Central

Wang, Lin-Cheng

2013-01-01

For most practical quantum control systems, it is important and difficult to attain robustness and reliability due to unavoidable uncertainties in the system dynamics or models. Three kinds of typical approaches (e.g., closed-loop learning control, feedback control, and robust control) have been proved to be effective to solve these problems. This work presents a self-contained survey on the closed-loop and robust control of quantum systems, as well as a brief introduction to a selection of basic theories and methods in this research area, to provide interested readers with a general idea for further studies. In the area of closed-loop learning control of quantum systems, we survey and introduce such learning control methods as gradient-based methods, genetic algorithms (GA), and reinforcement learning (RL) methods from a unified point of view of exploring the quantum control landscapes. For the feedback control approach, the paper surveys three control strategies including Lyapunov control, measurement-based control, and coherent-feedback control. Then such topics in the field of quantum robust control as H ∞ control, sliding mode control, quantum risk-sensitive control, and quantum ensemble control are reviewed. The paper concludes with a perspective of future research directions that are likely to attract more attention. PMID:23997680

Consensus-based distributed cooperative learning from closed-loop neural control systems.

PubMed

Chen, Weisheng; Hua, Shaoyong; Zhang, Huaguang

2015-02-01

In this paper, the neural tracking problem is addressed for a group of uncertain nonlinear systems where the system structures are identical but the reference signals are different. This paper focuses on studying the learning capability of neural networks (NNs) during the control process. First, we propose a novel control scheme called distributed cooperative learning (DCL) control scheme, by establishing the communication topology among adaptive laws of NN weights to share their learned knowledge online. It is further proved that if the communication topology is undirected and connected, all estimated weights of NNs can converge to small neighborhoods around their optimal values over a domain consisting of the union of all state orbits. Second, as a corollary it is shown that the conclusion on the deterministic learning still holds in the decentralized adaptive neural control scheme where, however, the estimated weights of NNs just converge to small neighborhoods of the optimal values along their own state orbits. Thus, the learned controllers obtained by DCL scheme have the better generalization capability than ones obtained by decentralized learning method. A simulation example is provided to verify the effectiveness and advantages of the control schemes proposed in this paper.

The Curse of Planning: Dissecting multiple reinforcement learning systems by taxing the central executive

PubMed Central

Otto, A. Ross; Gershman, Samuel J.; Markman, Arthur B.; Daw, Nathaniel D.

2013-01-01

A number of accounts of human and animal behavior posit the operation of parallel and competing valuation systems in the control of choice behavior. Along these lines, a flexible but computationally expensive model-based reinforcement learning system has been contrasted with a less flexible but more efficient model-free reinforcement learning system. The factors governing which system controls behavior—and under what circumstances—are still unclear. Based on the hypothesis that model-based reinforcement learning requires cognitive resources, we demonstrate that having human decision-makers perform a demanding secondary task engenders increased reliance on a model-free reinforcement learning strategy. Further, we show that across trials, people negotiate this tradeoff dynamically as a function of concurrent executive function demands and their choice latencies reflect the computational expenses of the strategy employed. These results demonstrate that competition between multiple learning systems can be controlled on a trial-by-trial basis by modulating the availability of cognitive resources. PMID:23558545

Fixed Point Learning Based Intelligent Traffic Control System

NASA Astrophysics Data System (ADS)

Zongyao, Wang; Cong, Sui; Cheng, Shao

2017-10-01

Fixed point learning has become an important tool to analyse large scale distributed system such as urban traffic network. This paper presents a fixed point learning based intelligence traffic network control system. The system applies convergence property of fixed point theorem to optimize the traffic flow density. The intelligence traffic control system achieves maximum road resources usage by averaging traffic flow density among the traffic network. The intelligence traffic network control system is built based on decentralized structure and intelligence cooperation. No central control is needed to manage the system. The proposed system is simple, effective and feasible for practical use. The performance of the system is tested via theoretical proof and simulations. The results demonstrate that the system can effectively solve the traffic congestion problem and increase the vehicles average speed. It also proves that the system is flexible, reliable and feasible for practical use.

A fuzzy classifier system for process control

NASA Technical Reports Server (NTRS)

Karr, C. L.; Phillips, J. C.

1994-01-01

A fuzzy classifier system that discovers rules for controlling a mathematical model of a pH titration system was developed by researchers at the U.S. Bureau of Mines (USBM). Fuzzy classifier systems successfully combine the strengths of learning classifier systems and fuzzy logic controllers. Learning classifier systems resemble familiar production rule-based systems, but they represent their IF-THEN rules by strings of characters rather than in the traditional linguistic terms. Fuzzy logic is a tool that allows for the incorporation of abstract concepts into rule based-systems, thereby allowing the rules to resemble the familiar 'rules-of-thumb' commonly used by humans when solving difficult process control and reasoning problems. Like learning classifier systems, fuzzy classifier systems employ a genetic algorithm to explore and sample new rules for manipulating the problem environment. Like fuzzy logic controllers, fuzzy classifier systems encapsulate knowledge in the form of production rules. The results presented in this paper demonstrate the ability of fuzzy classifier systems to generate a fuzzy logic-based process control system.

Fuzzy and neural control

NASA Technical Reports Server (NTRS)

Berenji, Hamid R.

1992-01-01

Fuzzy logic and neural networks provide new methods for designing control systems. Fuzzy logic controllers do not require a complete analytical model of a dynamic system and can provide knowledge-based heuristic controllers for ill-defined and complex systems. Neural networks can be used for learning control. In this chapter, we discuss hybrid methods using fuzzy logic and neural networks which can start with an approximate control knowledge base and refine it through reinforcement learning.

Experimental Verification of Electric Drive Technologies Based on Artificial Intelligence Tools

NASA Technical Reports Server (NTRS)

Rubaai, Ahmed; Ricketts, Daniel; Kotaru, Raj; Thomas, Robert; Noga, Donald F. (Technical Monitor); Kankam, Mark D. (Technical Monitor)

2000-01-01

In this report, a fully integrated prototype of a flight servo control system is successfully developed and implemented using brushless dc motors. The control system is developed by the fuzzy logic theory, and implemented with a multilayer neural network. First, a neural network-based architecture is introduced for fuzzy logic control. The characteristic rules and their membership functions of fuzzy systems are represented as the processing nodes in the neural network structure. The network structure and the parameter learning are performed simultaneously and online in the fuzzy-neural network system. The structure learning is based on the partition of input space. The parameter learning is based on the supervised gradient decent method, using a delta adaptation law. Using experimental setup, the performance of the proposed control system is evaluated under various operating conditions. Test results are presented and discussed in the report. The proposed learning control system has several advantages, namely, simple structure and learning capability, robustness and high tracking performance and few nodes at hidden layers. In comparison with the PI controller, the proposed fuzzy-neural network system can yield a better dynamic performance with shorter settling time, and without overshoot. Experimental results have shown that the proposed control system is adaptive and robust in responding to a wide range of operating conditions. In summary, the goal of this study is to design and implement-advanced servosystems to actuate control surfaces for flight vehicles, namely, aircraft and helicopters, missiles and interceptors, and mini- and micro-air vehicles.

Learner Control in Hypermedia Environments

ERIC Educational Resources Information Center

Scheiter, Katharina; Gerjets, Peter

2007-01-01

Contrary to system-controlled multimedia learning environments, hypermedia systems are characterized by a high level of interactivity. This interactivity is referred to as learner control in the respective literature. For several reasons this learner control is seen as a major advantage of hypermedia for learning and instruction. For instance,…

Model-Free Primitive-Based Iterative Learning Control Approach to Trajectory Tracking of MIMO Systems With Experimental Validation.

PubMed

Radac, Mircea-Bogdan; Precup, Radu-Emil; Petriu, Emil M

2015-11-01

This paper proposes a novel model-free trajectory tracking of multiple-input multiple-output (MIMO) systems by the combination of iterative learning control (ILC) and primitives. The optimal trajectory tracking solution is obtained in terms of previously learned solutions to simple tasks called primitives. The library of primitives that are stored in memory consists of pairs of reference input/controlled output signals. The reference input primitives are optimized in a model-free ILC framework without using knowledge of the controlled process. The guaranteed convergence of the learning scheme is built upon a model-free virtual reference feedback tuning design of the feedback decoupling controller. Each new complex trajectory to be tracked is decomposed into the output primitives regarded as basis functions. The optimal reference input for the control system to track the desired trajectory is next recomposed from the reference input primitives. This is advantageous because the optimal reference input is computed straightforward without the need to learn from repeated executions of the tracking task. In addition, the optimization problem specific to trajectory tracking of square MIMO systems is decomposed in a set of optimization problems assigned to each separate single-input single-output control channel that ensures a convenient model-free decoupling. The new model-free primitive-based ILC approach is capable of planning, reasoning, and learning. A case study dealing with the model-free control tuning for a nonlinear aerodynamic system is included to validate the new approach. The experimental results are given.

Using Control Theory to Teach Control Theory (or Any Other Course).

ERIC Educational Resources Information Center

Mansfield, George

1979-01-01

Describes an undergraduate automatic controls course in which the teaching-learning process is regarded as a closed loop feedback system. The three basic components of the system: the controller, the plant, and the learning sensors are identified as the teacher, the student, and student feedback respectively. (SMB)

Motor-response learning at a process control panel by an autonomous robot

DOE Office of Scientific and Technical Information (OSTI.GOV)

Spelt, P.F.; de Saussure, G.; Lyness, E.

1988-01-01

The Center for Engineering Systems Advanced Research (CESAR) was founded at Oak Ridge National Laboratory (ORNL) by the Department of Energy's Office of Energy Research/Division of Engineering and Geoscience (DOE-OER/DEG) to conduct basic research in the area of intelligent machines. Therefore, researchers at the CESAR Laboratory are engaged in a variety of research activities in the field of machine learning. In this paper, we describe our approach to a class of machine learning which involves motor response acquisition using feedback from trial-and-error learning. Our formulation is being experimentally validated using an autonomous robot, learning tasks of control panel monitoring andmore » manipulation for effect process control. The CLIPS Expert System and the associated knowledge base used by the robot in the learning process, which reside in a hypercube computer aboard the robot, are described in detail. Benchmark testing of the learning process on a robot/control panel simulation system consisting of two intercommunicating computers is presented, along with results of sample problems used to train and test the expert system. These data illustrate machine learning and the resulting performance improvement in the robot for problems similar to, but not identical with, those on which the robot was trained. Conclusions are drawn concerning the learning problems, and implications for future work on machine learning for autonomous robots are discussed. 16 refs., 4 figs., 1 tab.« less

Control of magnetic bearing systems via the Chebyshev polynomial-based unified model (CPBUM) neural network.

PubMed

Jeng, J T; Lee, T T

2000-01-01

A Chebyshev polynomial-based unified model (CPBUM) neural network is introduced and applied to control a magnetic bearing systems. First, we show that the CPBUM neural network not only has the same capability of universal approximator, but also has faster learning speed than conventional feedforward/recurrent neural network. It turns out that the CPBUM neural network is more suitable in the design of controller than the conventional feedforward/recurrent neural network. Second, we propose the inverse system method, based on the CPBUM neural networks, to control a magnetic bearing system. The proposed controller has two structures; namely, off-line and on-line learning structures. We derive a new learning algorithm for each proposed structure. The experimental results show that the proposed neural network architecture provides a greater flexibility and better performance in controlling magnetic bearing systems.

Learning from ISS-modular adaptive NN control of nonlinear strict-feedback systems.

PubMed

Wang, Cong; Wang, Min; Liu, Tengfei; Hill, David J

2012-10-01

This paper studies learning from adaptive neural control (ANC) for a class of nonlinear strict-feedback systems with unknown affine terms. To achieve the purpose of learning, a simple input-to-state stability (ISS) modular ANC method is first presented to ensure the boundedness of all the signals in the closed-loop system and the convergence of tracking errors in finite time. Subsequently, it is proven that learning with the proposed stable ISS-modular ANC can be achieved. The cascade structure and unknown affine terms of the considered systems make it very difficult to achieve learning using existing methods. To overcome these difficulties, the stable closed-loop system in the control process is decomposed into a series of linear time-varying (LTV) perturbed subsystems with the appropriate state transformation. Using a recursive design, the partial persistent excitation condition for the radial basis function neural network (NN) is established, which guarantees exponential stability of LTV perturbed subsystems. Consequently, accurate approximation of the closed-loop system dynamics is achieved in a local region along recurrent orbits of closed-loop signals, and learning is implemented during a closed-loop feedback control process. The learned knowledge is reused to achieve stability and an improved performance, thereby avoiding the tremendous repeated training process of NNs. Simulation studies are given to demonstrate the effectiveness of the proposed method.

Genetic algorithm based fuzzy control of spacecraft autonomous rendezvous

NASA Technical Reports Server (NTRS)

Karr, C. L.; Freeman, L. M.; Meredith, D. L.

1990-01-01

The U.S. Bureau of Mines is currently investigating ways to combine the control capabilities of fuzzy logic with the learning capabilities of genetic algorithms. Fuzzy logic allows for the uncertainty inherent in most control problems to be incorporated into conventional expert systems. Although fuzzy logic based expert systems have been used successfully for controlling a number of physical systems, the selection of acceptable fuzzy membership functions has generally been a subjective decision. High performance fuzzy membership functions for a fuzzy logic controller that manipulates a mathematical model simulating the autonomous rendezvous of spacecraft are learned using a genetic algorithm, a search technique based on the mechanics of natural genetics. The membership functions learned by the genetic algorithm provide for a more efficient fuzzy logic controller than membership functions selected by the authors for the rendezvous problem. Thus, genetic algorithms are potentially an effective and structured approach for learning fuzzy membership functions.

Patients with Parkinson's Disease Learn to Control Complex Systems via Procedural as Well as Non-Procedural Learning

ERIC Educational Resources Information Center

Osman, Magda; Wilkinson, Leonora; Beigi, Mazda; Castaneda, Cristina Sanchez; Jahanshahi, Marjan

2008-01-01

The striatum is considered to mediate some forms of procedural learning. Complex dynamic control (CDC) tasks involve an individual having to make a series of sequential decisions to achieve a specific outcome (e.g. learning to operate and control a car), and they involve procedural learning. The aim of this study was to test the hypothesis that…

Feedback Error Learning Controller for Functional Electrical Stimulation Assistance in a Hybrid Robotic System for Reaching Rehabilitation

PubMed Central

Resquín, Francisco; Gonzalez-Vargas, Jose; Ibáñez, Jaime; Brunetti, Fernando; Pons, José Luis

2016-01-01

Hybrid robotic systems represent a novel research field, where functional electrical stimulation (FES) is combined with a robotic device for rehabilitation of motor impairment. Under this approach, the design of robust FES controllers still remains an open challenge. In this work, we aimed at developing a learning FES controller to assist in the performance of reaching movements in a simple hybrid robotic system setting. We implemented a Feedback Error Learning (FEL) control strategy consisting of a feedback PID controller and a feedforward controller based on a neural network. A passive exoskeleton complemented the FES controller by compensating the effects of gravity. We carried out experiments with healthy subjects to validate the performance of the system. Results show that the FEL control strategy is able to adjust the FES intensity to track the desired trajectory accurately without the need of a previous mathematical model. PMID:27990245

Data-driven model reference control of MIMO vertical tank systems with model-free VRFT and Q-Learning.

PubMed

Radac, Mircea-Bogdan; Precup, Radu-Emil; Roman, Raul-Cristian

2018-02-01

This paper proposes a combined Virtual Reference Feedback Tuning-Q-learning model-free control approach, which tunes nonlinear static state feedback controllers to achieve output model reference tracking in an optimal control framework. The novel iterative Batch Fitted Q-learning strategy uses two neural networks to represent the value function (critic) and the controller (actor), and it is referred to as a mixed Virtual Reference Feedback Tuning-Batch Fitted Q-learning approach. Learning convergence of the Q-learning schemes generally depends, among other settings, on the efficient exploration of the state-action space. Handcrafting test signals for efficient exploration is difficult even for input-output stable unknown processes. Virtual Reference Feedback Tuning can ensure an initial stabilizing controller to be learned from few input-output data and it can be next used to collect substantially more input-state data in a controlled mode, in a constrained environment, by compensating the process dynamics. This data is used to learn significantly superior nonlinear state feedback neural networks controllers for model reference tracking, using the proposed Batch Fitted Q-learning iterative tuning strategy, motivating the original combination of the two techniques. The mixed Virtual Reference Feedback Tuning-Batch Fitted Q-learning approach is experimentally validated for water level control of a multi input-multi output nonlinear constrained coupled two-tank system. Discussions on the observed control behavior are offered. Copyright © 2018 ISA. Published by Elsevier Ltd. All rights reserved.

Cooperative learning neural network output feedback control of uncertain nonlinear multi-agent systems under directed topologies

NASA Astrophysics Data System (ADS)

Wang, W.; Wang, D.; Peng, Z. H.

2017-09-01

Without assuming that the communication topologies among the neural network (NN) weights are to be undirected and the states of each agent are measurable, the cooperative learning NN output feedback control is addressed for uncertain nonlinear multi-agent systems with identical structures in strict-feedback form. By establishing directed communication topologies among NN weights to share their learned knowledge, NNs with cooperative learning laws are employed to identify the uncertainties. By designing NN-based κ-filter observers to estimate the unmeasurable states, a new cooperative learning output feedback control scheme is proposed to guarantee that the system outputs can track nonidentical reference signals with bounded tracking errors. A simulation example is given to demonstrate the effectiveness of the theoretical results.

Feedback error learning control of magnetic satellites using type-2 fuzzy neural networks with elliptic membership functions.

PubMed

Khanesar, Mojtaba Ahmadieh; Kayacan, Erdal; Reyhanoglu, Mahmut; Kaynak, Okyay

2015-04-01

A novel type-2 fuzzy membership function (MF) in the form of an ellipse has recently been proposed in literature, the parameters of which that represent uncertainties are de-coupled from its parameters that determine the center and the support. This property has enabled the proposers to make an analytical comparison of the noise rejection capabilities of type-1 fuzzy logic systems with its type-2 counterparts. In this paper, a sliding mode control theory-based learning algorithm is proposed for an interval type-2 fuzzy logic system which benefits from elliptic type-2 fuzzy MFs. The learning is based on the feedback error learning method and not only the stability of the learning is proved but also the stability of the overall system is shown by adding an additional component to the control scheme to ensure robustness. In order to test the efficiency and efficacy of the proposed learning and the control algorithm, the trajectory tracking problem of a magnetic rigid spacecraft is studied. The simulations results show that the proposed control algorithm gives better performance results in terms of a smaller steady state error and a faster transient response as compared to conventional control algorithms.

Model-Free Optimal Tracking Control via Critic-Only Q-Learning.

PubMed

Luo, Biao; Liu, Derong; Huang, Tingwen; Wang, Ding

2016-10-01

Model-free control is an important and promising topic in control fields, which has attracted extensive attention in the past few years. In this paper, we aim to solve the model-free optimal tracking control problem of nonaffine nonlinear discrete-time systems. A critic-only Q-learning (CoQL) method is developed, which learns the optimal tracking control from real system data, and thus avoids solving the tracking Hamilton-Jacobi-Bellman equation. First, the Q-learning algorithm is proposed based on the augmented system, and its convergence is established. Using only one neural network for approximating the Q-function, the CoQL method is developed to implement the Q-learning algorithm. Furthermore, the convergence of the CoQL method is proved with the consideration of neural network approximation error. With the convergent Q-function obtained from the CoQL method, the adaptive optimal tracking control is designed based on the gradient descent scheme. Finally, the effectiveness of the developed CoQL method is demonstrated through simulation studies. The developed CoQL method learns with off-policy data and implements with a critic-only structure, thus it is easy to realize and overcome the inadequate exploration problem.

Laboratory Control System's Effects on Student Achievement and Attitudes

ERIC Educational Resources Information Center

Cicek, Fatma Gozalan; Taspinar, Mehmet

2016-01-01

Problem Statement: The current study investigates whether the learning environment designed based on the laboratory control system affects the academic achievement, the attitude toward the learning-teaching process and the retention of the students in computer education. Purpose of Study: The study aims to identify the laboratory control system…

Biomechanical Reconstruction Using the Tacit Learning System: Intuitive Control of Prosthetic Hand Rotation.

PubMed

Oyama, Shintaro; Shimoda, Shingo; Alnajjar, Fady S K; Iwatsuki, Katsuyuki; Hoshiyama, Minoru; Tanaka, Hirotaka; Hirata, Hitoshi

2016-01-01

Background: For mechanically reconstructing human biomechanical function, intuitive proportional control, and robustness to unexpected situations are required. Particularly, creating a functional hand prosthesis is a typical challenge in the reconstruction of lost biomechanical function. Nevertheless, currently available control algorithms are in the development phase. The most advanced algorithms for controlling multifunctional prosthesis are machine learning and pattern recognition of myoelectric signals. Despite the increase in computational speed, these methods cannot avoid the requirement of user consciousness and classified separation errors. "Tacit Learning System" is a simple but novel adaptive control strategy that can self-adapt its posture to environment changes. We introduced the strategy in the prosthesis rotation control to achieve compensatory reduction, as well as evaluated the system and its effects on the user. Methods: We conducted a non-randomized study involving eight prosthesis users to perform a bar relocation task with/without Tacit Learning System support. Hand piece and body motions were recorded continuously with goniometers, videos, and a motion-capture system. Findings: Reduction in the participants' upper extremity rotatory compensation motion was monitored during the relocation task in all participants. The estimated profile of total body energy consumption improved in five out of six participants. Interpretation: Our system rapidly accomplished nearly natural motion without unexpected errors. The Tacit Learning System not only adapts human motions but also enhances the human ability to adapt to the system quickly, while the system amplifies compensation generated by the residual limb. The concept can be extended to various situations for reconstructing lost functions that can be compensated.

Two Stages Cooperative Learning by Ability Indicators

ERIC Educational Resources Information Center

Wu, YuLung

2013-01-01

The teaching system in Taiwan is currently based on large classes where teachers cannot control student situations totally. In E-Learning System, a teacher who reviews a student's learning situation must examine the students' learning records according to different items, and further organize and define the students' current learning situations,…

A self-learning rule base for command following in dynamical systems

NASA Technical Reports Server (NTRS)

Tsai, Wei K.; Lee, Hon-Mun; Parlos, Alexander

1992-01-01

In this paper, a self-learning Rule Base for command following in dynamical systems is presented. The learning is accomplished though reinforcement learning using an associative memory called SAM. The main advantage of SAM is that it is a function approximator with explicit storage of training samples. A learning algorithm patterned after the dynamic programming is proposed. Two artificially created, unstable dynamical systems are used for testing, and the Rule Base was used to generate a feedback control to improve the command following ability of the otherwise uncontrolled systems. The numerical results are very encouraging. The controlled systems exhibit a more stable behavior and a better capability to follow reference commands. The rules resulting from the reinforcement learning are explicitly stored and they can be modified or augmented by human experts. Due to overlapping storage scheme of SAM, the stored rules are similar to fuzzy rules.

A neural network controller for automated composite manufacturing

NASA Technical Reports Server (NTRS)

Lichtenwalner, Peter F.

1994-01-01

At McDonnell Douglas Aerospace (MDA), an artificial neural network based control system has been developed and implemented to control laser heating for the fiber placement composite manufacturing process. This neurocontroller learns an approximate inverse model of the process on-line to provide performance that improves with experience and exceeds that of conventional feedback control techniques. When untrained, the control system behaves as a proportional plus integral (PI) controller. However after learning from experience, the neural network feedforward control module provides control signals that greatly improve temperature tracking performance. Faster convergence to new temperature set points and reduced temperature deviation due to changing feed rate have been demonstrated on the machine. A Cerebellar Model Articulation Controller (CMAC) network is used for inverse modeling because of its rapid learning performance. This control system is implemented in an IBM compatible 386 PC with an A/D board interface to the machine.

Refining fuzzy logic controllers with machine learning

NASA Technical Reports Server (NTRS)

Berenji, Hamid R.

1994-01-01

In this paper, we describe the GARIC (Generalized Approximate Reasoning-Based Intelligent Control) architecture, which learns from its past performance and modifies the labels in the fuzzy rules to improve performance. It uses fuzzy reinforcement learning which is a hybrid method of fuzzy logic and reinforcement learning. This technology can simplify and automate the application of fuzzy logic control to a variety of systems. GARIC has been applied in simulation studies of the Space Shuttle rendezvous and docking experiments. It has the potential of being applied in other aerospace systems as well as in consumer products such as appliances, cameras, and cars.

Filtering sensory information with XCSF: improving learning robustness and robot arm control performance.

PubMed

Kneissler, Jan; Stalph, Patrick O; Drugowitsch, Jan; Butz, Martin V

2014-01-01

It has been shown previously that the control of a robot arm can be efficiently learned using the XCSF learning classifier system, which is a nonlinear regression system based on evolutionary computation. So far, however, the predictive knowledge about how actual motor activity changes the state of the arm system has not been exploited. In this paper, we utilize the forward velocity kinematics knowledge of XCSF to alleviate the negative effect of noisy sensors for successful learning and control. We incorporate Kalman filtering for estimating successive arm positions, iteratively combining sensory readings with XCSF-based predictions of hand position changes over time. The filtered arm position is used to improve both trajectory planning and further learning of the forward velocity kinematics. We test the approach on a simulated kinematic robot arm model. The results show that the combination can improve learning and control performance significantly. However, it also shows that variance estimates of XCSF prediction may be underestimated, in which case self-delusional spiraling effects can hinder effective learning. Thus, we introduce a heuristic parameter, which can be motivated by theory, and which limits the influence of XCSF's predictions on its own further learning input. As a result, we obtain drastic improvements in noise tolerance, allowing the system to cope with more than 10 times higher noise levels.

Controlled Experiment Replication in Evaluation of E-Learning System's Educational Influence

ERIC Educational Resources Information Center

Grubisic, Ani; Stankov, Slavomir; Rosic, Marko; Zitko, Branko

2009-01-01

We believe that every effectiveness evaluation should be replicated at least in order to verify the original results and to indicate evaluated e-learning system's advantages or disadvantages. This paper presents the methodology for conducting controlled experiment replication, as well as, results of a controlled experiment and an internal…

Data-Driven H∞ Control for Nonlinear Distributed Parameter Systems.

PubMed

Luo, Biao; Huang, Tingwen; Wu, Huai-Ning; Yang, Xiong

2015-11-01

The data-driven H∞ control problem of nonlinear distributed parameter systems is considered in this paper. An off-policy learning method is developed to learn the H∞ control policy from real system data rather than the mathematical model. First, Karhunen-Loève decomposition is used to compute the empirical eigenfunctions, which are then employed to derive a reduced-order model (ROM) of slow subsystem based on the singular perturbation theory. The H∞ control problem is reformulated based on the ROM, which can be transformed to solve the Hamilton-Jacobi-Isaacs (HJI) equation, theoretically. To learn the solution of the HJI equation from real system data, a data-driven off-policy learning approach is proposed based on the simultaneous policy update algorithm and its convergence is proved. For implementation purpose, a neural network (NN)- based action-critic structure is developed, where a critic NN and two action NNs are employed to approximate the value function, control, and disturbance policies, respectively. Subsequently, a least-square NN weight-tuning rule is derived with the method of weighted residuals. Finally, the developed data-driven off-policy learning approach is applied to a nonlinear diffusion-reaction process, and the obtained results demonstrate its effectiveness.

Scheduled power tracking control of the wind-storage hybrid system based on the reinforcement learning theory

NASA Astrophysics Data System (ADS)

Li, Ze

2017-09-01

In allusion to the intermittency and uncertainty of the wind electricity, energy storage and wind generator are combined into a hybrid system to improve the controllability of the output power. A scheduled power tracking control method is proposed based on the reinforcement learning theory and Q-learning algorithm. In this method, the state space of the environment is formed with two key factors, i.e. the state of charge of the energy storage and the difference value between the actual wind power and scheduled power, the feasible action is the output power of the energy storage, and the corresponding immediate rewarding function is designed to reflect the rationality of the control action. By interacting with the environment and learning from the immediate reward, the optimal control strategy is gradually formed. After that, it could be applied to the scheduled power tracking control of the hybrid system. Finally, the rationality and validity of the method are verified through simulation examples.

Development and training of a learning expert system in an autonomous mobile robot via simulation

DOE Office of Scientific and Technical Information (OSTI.GOV)

Spelt, P.F.; Lyness, E.; DeSaussure, G.

1989-11-01

The Center for Engineering Systems Advanced Research (CESAR) conducts basic research in the area of intelligent machines. Recently at CESAR a learning expert system was created to operate on board an autonomous robot working at a process control panel. The authors discuss two-computer simulation system used to create, evaluate and train this learning system. The simulation system has a graphics display of the current status of the process being simulated, and the same program which does the simulating also drives the actual control panel. Simulation results were validated on the actual robot. The speed and safety values of using amore » computerized simulator to train a learning computer, and future uses of the simulation system, are discussed.« less

Intelligent power management in a vehicular system with multiple power sources

NASA Astrophysics Data System (ADS)

Murphey, Yi L.; Chen, ZhiHang; Kiliaris, Leonidas; Masrur, M. Abul

This paper presents an optimal online power management strategy applied to a vehicular power system that contains multiple power sources and deals with largely fluctuated load requests. The optimal online power management strategy is developed using machine learning and fuzzy logic. A machine learning algorithm has been developed to learn the knowledge about minimizing power loss in a Multiple Power Sources and Loads (M_PS&LD) system. The algorithm exploits the fact that different power sources used to deliver a load request have different power losses under different vehicle states. The machine learning algorithm is developed to train an intelligent power controller, an online fuzzy power controller, FPC_MPS, that has the capability of finding combinations of power sources that minimize power losses while satisfying a given set of system and component constraints during a drive cycle. The FPC_MPS was implemented in two simulated systems, a power system of four power sources, and a vehicle system of three power sources. Experimental results show that the proposed machine learning approach combined with fuzzy control is a promising technology for intelligent vehicle power management in a M_PS&LD power system.

REVIEW: Internal models in sensorimotor integration: perspectives from adaptive control theory

NASA Astrophysics Data System (ADS)

Tin, Chung; Poon, Chi-Sang

2005-09-01

Internal models and adaptive controls are empirical and mathematical paradigms that have evolved separately to describe learning control processes in brain systems and engineering systems, respectively. This paper presents a comprehensive appraisal of the correlation between these paradigms with a view to forging a unified theoretical framework that may benefit both disciplines. It is suggested that the classic equilibrium-point theory of impedance control of arm movement is analogous to continuous gain-scheduling or high-gain adaptive control within or across movement trials, respectively, and that the recently proposed inverse internal model is akin to adaptive sliding control originally for robotic manipulator applications. Modular internal models' architecture for multiple motor tasks is a form of multi-model adaptive control. Stochastic methods, such as generalized predictive control, reinforcement learning, Bayesian learning and Hebbian feedback covariance learning, are reviewed and their possible relevance to motor control is discussed. Possible applicability of a Luenberger observer and an extended Kalman filter to state estimation problems—such as sensorimotor prediction or the resolution of vestibular sensory ambiguity—is also discussed. The important role played by vestibular system identification in postural control suggests an indirect adaptive control scheme whereby system states or parameters are explicitly estimated prior to the implementation of control. This interdisciplinary framework should facilitate the experimental elucidation of the mechanisms of internal models in sensorimotor systems and the reverse engineering of such neural mechanisms into novel brain-inspired adaptive control paradigms in future.

An e-Learning System with MR for Experiments Involving Circuit Construction to Control a Robot

ERIC Educational Resources Information Center

Takemura, Atsushi

2016-01-01

This paper proposes a novel e-Learning system for technological experiments involving electronic circuit-construction and controlling robot motion that are necessary in the field of technology. The proposed system performs automated recognition of circuit images transmitted from individual learners and automatically supplies the learner with…

Motor Task Variation Induces Structural Learning

PubMed Central

Braun, Daniel A.; Aertsen, Ad; Wolpert, Daniel M.; Mehring, Carsten

2009-01-01

Summary When we have learned a motor skill, such as cycling or ice-skating, we can rapidly generalize to novel tasks, such as motorcycling or rollerblading [1–8]. Such facilitation of learning could arise through two distinct mechanisms by which the motor system might adjust its control parameters. First, fast learning could simply be a consequence of the proximity of the original and final settings of the control parameters. Second, by structural learning [9–14], the motor system could constrain the parameter adjustments to conform to the control parameters' covariance structure. Thus, facilitation of learning would rely on the novel task parameters' lying on the structure of a lower-dimensional subspace that can be explored more efficiently. To test between these two hypotheses, we exposed subjects to randomly varying visuomotor tasks of fixed structure. Although such randomly varying tasks are thought to prevent learning, we show that when subsequently presented with novel tasks, subjects exhibit three key features of structural learning: facilitated learning of tasks with the same structure, strong reduction in interference normally observed when switching between tasks that require opposite control strategies, and preferential exploration along the learned structure. These results suggest that skill generalization relies on task variation and structural learning. PMID:19217296

Motor task variation induces structural learning.

PubMed

Braun, Daniel A; Aertsen, Ad; Wolpert, Daniel M; Mehring, Carsten

2009-02-24

When we have learned a motor skill, such as cycling or ice-skating, we can rapidly generalize to novel tasks, such as motorcycling or rollerblading [1-8]. Such facilitation of learning could arise through two distinct mechanisms by which the motor system might adjust its control parameters. First, fast learning could simply be a consequence of the proximity of the original and final settings of the control parameters. Second, by structural learning [9-14], the motor system could constrain the parameter adjustments to conform to the control parameters' covariance structure. Thus, facilitation of learning would rely on the novel task parameters' lying on the structure of a lower-dimensional subspace that can be explored more efficiently. To test between these two hypotheses, we exposed subjects to randomly varying visuomotor tasks of fixed structure. Although such randomly varying tasks are thought to prevent learning, we show that when subsequently presented with novel tasks, subjects exhibit three key features of structural learning: facilitated learning of tasks with the same structure, strong reduction in interference normally observed when switching between tasks that require opposite control strategies, and preferential exploration along the learned structure. These results suggest that skill generalization relies on task variation and structural learning.

Importance Of Quality Control in Reducing System Risk, a Lesson Learned From The Shuttle and a Recommendation for Future Launch Vehicles

NASA Technical Reports Server (NTRS)

Safie, Fayssal M.; Messer, Bradley P.

2006-01-01

This paper presents lessons learned from the Space Shuttle return to flight experience and the importance of these lessons learned in the development of new the NASA Crew Launch Vehicle (CLV). Specifically, the paper discusses the relationship between process control and system risk, and the importance of process control in improving space vehicle flight safety. It uses the External Tank (ET) Thermal Protection System (TPS) experience and lessons learned from the redesign and process enhancement activities performed in preparation for Return to Flight after the Columbia accident. The paper also, discusses in some details, the Probabilistic engineering physics based risk assessment performed by the Shuttle program to evaluate the impact of TPS failure on system risk and the application of the methodology to the CLV.

Adaptive Computerized Instruction.

ERIC Educational Resources Information Center

Ray, Roger D.; And Others

1995-01-01

Describes an artificially intelligent multimedia computerized instruction system capable of developing a conceptual image of what a student is learning while the student is learning it. It focuses on principles of learning and adaptive behavioral control systems theory upon which the system is designed and demonstrates multiple user modes.…

Parallel Online Temporal Difference Learning for Motor Control.

PubMed

Caarls, Wouter; Schuitema, Erik

2016-07-01

Temporal difference (TD) learning, a key concept in reinforcement learning, is a popular method for solving simulated control problems. However, in real systems, this method is often avoided in favor of policy search methods because of its long learning time. But policy search suffers from its own drawbacks, such as the necessity of informed policy parameterization and initialization. In this paper, we show that TD learning can work effectively in real robotic systems as well, using parallel model learning and planning. Using locally weighted linear regression and trajectory sampled planning with 14 concurrent threads, we can achieve a speedup of almost two orders of magnitude over regular TD control on simulated control benchmarks. For a real-world pendulum swing-up task and a two-link manipulator movement task, we report a speedup of 20× to 60× , with a real-time learning speed of less than half a minute. The results are competitive with state-of-the-art policy search.

Compensation of significant parametric uncertainties using sliding mode online learning

NASA Astrophysics Data System (ADS)

Schnetter, Philipp; Kruger, Thomas

An augmented nonlinear inverse dynamics (NID) flight control strategy using sliding mode online learning for a small unmanned aircraft system (UAS) is presented. Because parameter identification for this class of aircraft often is not valid throughout the complete flight envelope, aerodynamic parameters used for model based control strategies may show significant deviations. For the concept of feedback linearization this leads to inversion errors that in combination with the distinctive susceptibility of small UAS towards atmospheric turbulence pose a demanding control task for these systems. In this work an adaptive flight control strategy using feedforward neural networks for counteracting such nonlinear effects is augmented with the concept of sliding mode control (SMC). SMC-learning is derived from variable structure theory. It considers a neural network and its training as a control problem. It is shown that by the dynamic calculation of the learning rates, stability can be guaranteed and thus increase the robustness against external disturbances and system failures. With the resulting higher speed of convergence a wide range of simultaneously occurring disturbances can be compensated. The SMC-based flight controller is tested and compared to the standard gradient descent (GD) backpropagation algorithm under the influence of significant model uncertainties and system failures.

LMS in university for in-class education: Synergy of free software, competitive approach and social networks technology

NASA Astrophysics Data System (ADS)

Radygin, V. Y.; Lukyanova, N. V.; Kupriyanov, D. Yu.

2017-01-01

Transformation of learning management systems over last two decades was investigated. The features of using e-learning systems for in-class education were discussed. The necessity of integration e-learning system with the student performance controlling system was shown. The influence of choice of student ranking system on students' motivation was described. The own way to choice of e-learning system design principles and technologies were suggested.

Parameter learning for performance adaptation

NASA Technical Reports Server (NTRS)

Peek, Mark D.; Antsaklis, Panos J.

1990-01-01

A parameter learning method is introduced and used to broaden the region of operability of the adaptive control system of a flexible space antenna. The learning system guides the selection of control parameters in a process leading to optimal system performance. A grid search procedure is used to estimate an initial set of parameter values. The optimization search procedure uses a variation of the Hooke and Jeeves multidimensional search algorithm. The method is applicable to any system where performance depends on a number of adjustable parameters. A mathematical model is not necessary, as the learning system can be used whenever the performance can be measured via simulation or experiment. The results of two experiments, the transient regulation and the command following experiment, are presented.

Patients with Parkinson's disease learn to control complex systems-an indication for intact implicit cognitive skill learning.

PubMed

Witt, Karsten; Daniels, Christine; Daniel, Victoria; Schmitt-Eliassen, Julia; Volkmann, Jens; Deuschl, Günther

2006-01-01

Implicit memory and learning mechanisms are composed of multiple processes and systems. Previous studies demonstrated a basal ganglia involvement in purely cognitive tasks that form stimulus response habits by reinforcement learning such as implicit classification learning. We will test the basal ganglia influence on two cognitive implicit tasks previously described by Berry and Broadbent, the sugar production task and the personal interaction task. Furthermore, we will investigate the relationship between certain aspects of an executive dysfunction and implicit learning. To this end, we have tested 22 Parkinsonian patients and 22 age-matched controls on two implicit cognitive tasks, in which participants learned to control a complex system. They interacted with the system by choosing an input value and obtaining an output that was related in a complex manner to the input. The objective was to reach and maintain a specific target value across trials (dynamic system learning). The two tasks followed the same underlying complex rule but had different surface appearances. Subsequently, participants performed an executive test battery including the Stroop test, verbal fluency and the Wisconsin card sorting test (WCST). The results demonstrate intact implicit learning in patients, despite an executive dysfunction in the Parkinsonian group. They lead to the conclusion that the basal ganglia system affected in Parkinson's disease does not contribute to the implicit acquisition of a new cognitive skill. Furthermore, the Parkinsonian patients were able to reach a specific goal in an implicit learning context despite impaired goal directed behaviour in the WCST, a classic test of executive functions. These results demonstrate a functional independence of implicit cognitive skill learning and certain aspects of executive functions.

OLMS: Online Learning Management System for E-Learning

ERIC Educational Resources Information Center

Ippakayala, Vinay Kumar; El-Ocla, Hosam

2017-01-01

In this paper we introduce a learning management system that provides a management system for centralized control of course content. A secure system to record lectures is implemented as a key feature of this application. This feature would be accessed through web camera and mobile recording. These features are mainly designed for e-learning…

Machine Learning Control For Highly Reconfigurable High-Order Systems

DTIC Science & Technology

2015-01-02

develop and flight test a Reinforcement Learning based approach for autonomous tracking of ground targets using a fixed wing Unmanned...Reinforcement Learning - based algorithms are developed for learning agents’ time dependent dynamics while also learning to control them. Three algorithms...to a wide range of engineering- based problems . Implementation of these solutions, however, is often complicated by the hysteretic, non-linear,

Adaptive learning and control for MIMO system based on adaptive dynamic programming.

PubMed

Fu, Jian; He, Haibo; Zhou, Xinmin

2011-07-01

Adaptive dynamic programming (ADP) is a promising research field for design of intelligent controllers, which can both learn on-the-fly and exhibit optimal behavior. Over the past decades, several generations of ADP design have been proposed in the literature, which have demonstrated many successful applications in various benchmarks and industrial applications. While many of the existing researches focus on multiple-inputs-single-output system with steepest descent search, in this paper we investigate a generalized multiple-input-multiple-output (GMIMO) ADP design for online learning and control, which is more applicable to a wide range of practical real-world applications. Furthermore, an improved weight-updating algorithm based on recursive Levenberg-Marquardt methods is presented and embodied in the GMIMO approach to improve its performance. Finally, we test the performance of this approach based on a practical complex system, namely, the learning and control of the tension and height of the looper system in a hot strip mill. Experimental results demonstrate that the proposed approach can achieve effective and robust performance.

Multi Car Elevator Control by using Learning Automaton

NASA Astrophysics Data System (ADS)

Shiraishi, Kazuaki; Hamagami, Tomoki; Hirata, Hironori

We study an adaptive control technique for multi car elevators (MCEs) by adopting learning automatons (LAs.) The MCE is a high performance and a near-future elevator system with multi shafts and multi cars. A strong point of the system is that realizing a large carrying capacity in small shaft area. However, since the operation is too complicated, realizing an efficient MCE control is difficult for top-down approaches. For example, “bunching up together" is one of the typical phenomenon in a simple traffic environment like the MCE. Furthermore, an adapting to varying environment in configuration requirement is a serious issue in a real elevator service. In order to resolve these issues, having an autonomous behavior is required to the control system of each car in MCE system, so that the learning automaton, as the solutions for this requirement, is supposed to be appropriate for the simple traffic control. First, we assign a stochastic automaton (SA) to each car control system. Then, each SA varies its stochastic behavior distributions for adapting to environment in which its policy is evaluated with each passenger waiting times. That is LA which learns the environment autonomously. Using the LA based control technique, the MCE operation efficiency is evaluated through simulation experiments. Results show the technique enables reducing waiting times efficiently, and we confirm the system can adapt to the dynamic environment.

Optimal critic learning for robot control in time-varying environments.

PubMed

Wang, Chen; Li, Yanan; Ge, Shuzhi Sam; Lee, Tong Heng

2015-10-01

In this paper, optimal critic learning is developed for robot control in a time-varying environment. The unknown environment is described as a linear system with time-varying parameters, and impedance control is employed for the interaction control. Desired impedance parameters are obtained in the sense of an optimal realization of the composite of trajectory tracking and force regulation. Q -function-based critic learning is developed to determine the optimal impedance parameters without the knowledge of the system dynamics. The simulation results are presented and compared with existing methods, and the efficacy of the proposed method is verified.

Facts and fiction of learning systems. [decision making intelligent control

NASA Technical Reports Server (NTRS)

Saridis, G. N.

1975-01-01

The methodology that will provide the updated precision for the hardware control and the advanced decision making and planning in the software control is called learning systems and intelligent control. It was developed theoretically as an alternative for the nonsystematic heuristic approaches of artificial intelligence experiments and the inflexible formulation of modern optimal control methods. Its basic concepts are discussed and some feasibility studies of some practical applications are presented.

Lessons Learned and Flight Results from the F15 Intelligent Flight Control System Project

NASA Technical Reports Server (NTRS)

Bosworth, John

2006-01-01

A viewgraph presentation on the lessons learned and flight results from the F15 Intelligent Flight Control System (IFCS) project is shown. The topics include: 1) F-15 IFCS Project Goals; 2) Motivation; 3) IFCS Approach; 4) NASA F-15 #837 Aircraft Description; 5) Flight Envelope; 6) Limited Authority System; 7) NN Floating Limiter; 8) Flight Experiment; 9) Adaptation Goals; 10) Handling Qualities Performance Metric; 11) Project Phases; 12) Indirect Adaptive Control Architecture; 13) Indirect Adaptive Experience and Lessons Learned; 14) Gen II Direct Adaptive Control Architecture; 15) Current Status; 16) Effect of Canard Multiplier; 17) Simulated Canard Failure Stab Open Loop; 18) Canard Multiplier Effect Closed Loop Freq. Resp.; 19) Simulated Canard Failure Stab Open Loop with Adaptation; 20) Canard Multiplier Effect Closed Loop with Adaptation; 21) Gen 2 NN Wts from Simulation; 22) Direct Adaptive Experience and Lessons Learned; and 23) Conclusions

Adaptive iterative learning control of a class of nonlinear time-delay systems with unknown backlash-like hysteresis input and control direction.

PubMed

Wei, Jianming; Zhang, Youan; Sun, Meimei; Geng, Baoliang

2017-09-01

This paper presents an adaptive iterative learning control scheme for a class of nonlinear systems with unknown time-varying delays and control direction preceded by unknown nonlinear backlash-like hysteresis. Boundary layer function is introduced to construct an auxiliary error variable, which relaxes the identical initial condition assumption of iterative learning control. For the controller design, integral Lyapunov function candidate is used, which avoids the possible singularity problem by introducing hyperbolic tangent funciton. After compensating for uncertainties with time-varying delays by combining appropriate Lyapunov-Krasovskii function with Young's inequality, an adaptive iterative learning control scheme is designed through neural approximation technique and Nussbaum function method. On the basis of the hyperbolic tangent function's characteristics, the system output is proved to converge to a small neighborhood of the desired trajectory by constructing Lyapunov-like composite energy function (CEF) in two cases, while keeping all the closed-loop signals bounded. Finally, a simulation example is presented to verify the effectiveness of the proposed approach. Copyright © 2017 ISA. Published by Elsevier Ltd. All rights reserved.

Reinforcement learning of periodical gaits in locomotion robots

NASA Astrophysics Data System (ADS)

Svinin, Mikhail; Yamada, Kazuyaki; Ushio, S.; Ueda, Kanji

1999-08-01

Emergence of stable gaits in locomotion robots is studied in this paper. A classifier system, implementing an instance- based reinforcement learning scheme, is used for sensory- motor control of an eight-legged mobile robot. Important feature of the classifier system is its ability to work with the continuous sensor space. The robot does not have a prior knowledge of the environment, its own internal model, and the goal coordinates. It is only assumed that the robot can acquire stable gaits by learning how to reach a light source. During the learning process the control system, is self-organized by reinforcement signals. Reaching the light source defines a global reward. Forward motion gets a local reward, while stepping back and falling down get a local punishment. Feasibility of the proposed self-organized system is tested under simulation and experiment. The control actions are specified at the leg level. It is shown that, as learning progresses, the number of the action rules in the classifier systems is stabilized to a certain level, corresponding to the acquired gait patterns.

Integration of Online Parameter Identification and Neural Network for In-Flight Adaptive Control

NASA Technical Reports Server (NTRS)

Hageman, Jacob J.; Smith, Mark S.; Stachowiak, Susan

2003-01-01

An indirect adaptive system has been constructed for robust control of an aircraft with uncertain aerodynamic characteristics. This system consists of a multilayer perceptron pre-trained neural network, online stability and control derivative identification, a dynamic cell structure online learning neural network, and a model following control system based on the stochastic optimal feedforward and feedback technique. The pre-trained neural network and model following control system have been flight-tested, but the online parameter identification and online learning neural network are new additions used for in-flight adaptation of the control system model. A description of the modification and integration of these two stand-alone software packages into the complete system in preparation for initial flight tests is presented. Open-loop results using both simulation and flight data, as well as closed-loop performance of the complete system in a nonlinear, six-degree-of-freedom, flight validated simulation, are analyzed. Results show that this online learning system, in contrast to the nonlearning system, has the ability to adapt to changes in aerodynamic characteristics in a real-time, closed-loop, piloted simulation, resulting in improved flying qualities.

Fuzzy controller training using particle swarm optimization for nonlinear system control.

PubMed

Karakuzu, Cihan

2008-04-01

This paper proposes and describes an effective utilization of particle swarm optimization (PSO) to train a Takagi-Sugeno (TS)-type fuzzy controller. Performance evaluation of the proposed fuzzy training method using the obtained simulation results is provided with two samples of highly nonlinear systems: a continuous stirred tank reactor (CSTR) and a Van der Pol (VDP) oscillator. The superiority of the proposed learning technique is that there is no need for a partial derivative with respect to the parameter for learning. This fuzzy learning technique is suitable for real-time implementation, especially if the system model is unknown and a supervised training cannot be run. In this study, all parameters of the controller are optimized with PSO in order to prove that a fuzzy controller trained by PSO exhibits a good control performance.

Model-Free control performance improvement using virtual reference feedback tuning and reinforcement Q-learning

NASA Astrophysics Data System (ADS)

Radac, Mircea-Bogdan; Precup, Radu-Emil; Roman, Raul-Cristian

2017-04-01

This paper proposes the combination of two model-free controller tuning techniques, namely linear virtual reference feedback tuning (VRFT) and nonlinear state-feedback Q-learning, referred to as a new mixed VRFT-Q learning approach. VRFT is first used to find stabilising feedback controller using input-output experimental data from the process in a model reference tracking setting. Reinforcement Q-learning is next applied in the same setting using input-state experimental data collected under perturbed VRFT to ensure good exploration. The Q-learning controller learned with a batch fitted Q iteration algorithm uses two neural networks, one for the Q-function estimator and one for the controller, respectively. The VRFT-Q learning approach is validated on position control of a two-degrees-of-motion open-loop stable multi input-multi output (MIMO) aerodynamic system (AS). Extensive simulations for the two independent control channels of the MIMO AS show that the Q-learning controllers clearly improve performance over the VRFT controllers.

Formation Learning Control of Multiple Autonomous Underwater Vehicles With Heterogeneous Nonlinear Uncertain Dynamics.

PubMed

Yuan, Chengzhi; Licht, Stephen; He, Haibo

2017-09-26

In this paper, a new concept of formation learning control is introduced to the field of formation control of multiple autonomous underwater vehicles (AUVs), which specifies a joint objective of distributed formation tracking control and learning/identification of nonlinear uncertain AUV dynamics. A novel two-layer distributed formation learning control scheme is proposed, which consists of an upper-layer distributed adaptive observer and a lower-layer decentralized deterministic learning controller. This new formation learning control scheme advances existing techniques in three important ways: 1) the multi-AUV system under consideration has heterogeneous nonlinear uncertain dynamics; 2) the formation learning control protocol can be designed and implemented by each local AUV agent in a fully distributed fashion without using any global information; and 3) in addition to the formation control performance, the distributed control protocol is also capable of accurately identifying the AUVs' heterogeneous nonlinear uncertain dynamics and utilizing experiences to improve formation control performance. Extensive simulations have been conducted to demonstrate the effectiveness of the proposed results.

Mind map learning for advanced engineering study: case study in system dynamics

NASA Astrophysics Data System (ADS)

Woradechjumroen, Denchai

2018-01-01

System Dynamics (SD) is one of the subjects that were use in learning Automatic Control Systems in dynamic and control field. Mathematical modelling and solving skills of students for engineering systems are expecting outcomes of the course which can be further used to efficiently study control systems and mechanical vibration; however, the fundamental of the SD includes strong backgrounds in Dynamics and Differential Equations, which are appropriate to the students in governmental universities that have strong skills in Mathematics and Scientifics. For private universities, students are weak in the above subjects since they obtained high vocational certificate from Technical College or Polytechnic School, which emphasize the learning contents in practice. To enhance their learning for improving their backgrounds, this paper applies mind maps based problem based learning to relate the essential relations of mathematical and physical equations. With the advantages of mind maps, each student is assigned to design individual mind maps for self-leaning development after they attend the class and learn overall picture of each chapter from the class instructor. Four problems based mind maps learning are assigned to each student. Each assignment is evaluated via mid-term and final examinations, which are issued in terms of learning concepts and applications. In the method testing, thirty students are tested and evaluated via student learning backgrounds in the past. The result shows that well-design mind maps can improve learning performance based on outcome evaluation. Especially, mind maps can reduce time-consuming and reviewing for Mathematics and Physics in SD significantly.

Visuomotor coordination and cortical connectivity of modular motor learning.

PubMed

Burgos, Pablo I; Mariman, Juan J; Makeig, Scott; Rivera-Lillo, Gonzalo; Maldonado, Pedro E

2018-05-15

The ability to transfer sensorimotor skill components to new actions and the capacity to use skill components from whole actions are characteristic of the adaptability of the human sensorimotor system. However, behavioral evidence suggests complex limitations for transfer after combined or modular learning of motor adaptations. Also, to date, only behavioral analysis of the consequences of the modular learning has been reported, with little understanding of the sensorimotor mechanisms of control and the interaction between cortical areas. We programmed a video game with distorted kinematic and dynamic features to test the ability to combine sensorimotor skill components learned modularly (composition) and the capacity to use separate sensorimotor skill components learned in combination (decomposition). We examined motor performance, eye-hand coordination, and EEG connectivity. When tested for integrated learning, we found that combined practice initially performed better than separated practice, but differences disappeared after integrated practice. Separate learning promotes fewer anticipatory control mechanisms (depending more on feedback control), evidenced in a lower gaze leading behavior and in higher connectivity between visual and premotor domains, in comparison with the combined practice. The sensorimotor system can acquire motor modules in a separated or integrated manner. However, the system appears to require integrated practice to coordinate the adaptations with the skill learning and the networks involved in the integrated behavior. This integration seems to be related to the acquisition of anticipatory mechanism of control and with the decrement of feedback control. © 2018 Wiley Periodicals, Inc.

An architecture for designing fuzzy logic controllers using neural networks

NASA Technical Reports Server (NTRS)

Berenji, Hamid R.

1991-01-01

Described here is an architecture for designing fuzzy controllers through a hierarchical process of control rule acquisition and by using special classes of neural network learning techniques. A new method for learning to refine a fuzzy logic controller is introduced. A reinforcement learning technique is used in conjunction with a multi-layer neural network model of a fuzzy controller. The model learns by updating its prediction of the plant's behavior and is related to the Sutton's Temporal Difference (TD) method. The method proposed here has the advantage of using the control knowledge of an experienced operator and fine-tuning it through the process of learning. The approach is applied to a cart-pole balancing system.

The neurobiology of safety and threat learning in infancy.

PubMed

Debiec, Jacek; Sullivan, Regina M

2017-09-01

What an animal needs to learn to survive is altered dramatically as they change from dependence on the parent for protection to independence and reliance on self-defense. This transition occurs in most altricial animals, but our understanding of the behavioral neurobiology has mostly relied on the infant rat. The transformation from dependence to independence occurs over three weeks in pups and is accompanied by complex changes in responses to both natural and learned threats and the supporting neural circuitry. Overall, in early life, the threat system is quiescent and learning is biased towards acquiring attachment related behaviors to support attachment to the caregiver and proximity seeking. Caregiver-associated cues learned in infancy have the ability to provide a sense of safety throughout lifetime. This attachment/safety system is activated by learning involving presumably pleasurable stimuli (food, warmth) but also painful stimuli (tailpinch, moderate shock). At about the midway point to independence, pups begin to have access to the adult-like amygdala-dependent threat system and amygdala-dependent responses to natural dangers such as predator odors. However, pups have the ability to switch between the infant and adult-like system, which is controlled by maternal presence and modification of stress hormones. Specifically, if the pup is alone, it will learn fear but if with the mother it will learn attachment (10-15days of age). As pups begin to approach weaning, pups lose access to the attachment system and rely only on the amygdala-dependent threat system. However, pups learning system is complex and exhibits flexibility that enables the mother to override the control of the attachment circuit, since newborn pups may acquire threat responses from the mother expressing fear in their presence. Together, these data suggest that the development of pups' threat learning system is not only dependent upon maturation of the amygdala, but it is also exquisitely controlled by the environment. Most notably the mother can switch pup learning between attachment to threat learning in a moment's notice. This enables the mother to navigate pup's learning about the world and what is threatening and what is safe. Copyright © 2016 Elsevier Inc. All rights reserved.

Discrete time learning control in nonlinear systems

NASA Technical Reports Server (NTRS)

Longman, Richard W.; Chang, Chi-Kuang; Phan, Minh

1992-01-01

In this paper digital learning control methods are developed primarily for use in single-input, single-output nonlinear dynamic systems. Conditions for convergence of the basic form of learning control based on integral control concepts are given, and shown to be satisfied by a large class of nonlinear problems. It is shown that it is not the gross nonlinearities of the differential equations that matter in the convergence, but rather the much smaller nonlinearities that can manifest themselves during the short time interval of one sample time. New algorithms are developed that eliminate restrictions on the size of the learning gain, and on knowledge of the appropriate sign of the learning gain, for convergence to zero error in tracking a feasible desired output trajectory. It is shown that one of the new algorithms can give guaranteed convergence in the presence of actuator saturation constraints, and indicate when the requested trajectory is beyond the actuator capabilities.

Models of Learning in ICAI.

ERIC Educational Resources Information Center

Duchastel, P.; And Others

1989-01-01

Discusses intelligent computer assisted instruction (ICAI) and presents various models of learning which have been proposed. Topics discussed include artificial intelligence; intelligent tutorial systems; tutorial strategies; learner control; system design; learning theory; and knowledge representation of proper and improper (i.e., incorrect)…

Implementation of Project Based Learning in Mechatronic Lab Course at Bandung State Polytechnic

ERIC Educational Resources Information Center

Basjaruddin, Noor Cholis; Rakhman, Edi

2016-01-01

Mechatronics is a multidisciplinary that includes a combination of mechanics, electronics, control systems, and computer science. The main objective of mechatronics learning is to establish a comprehensive mindset in the development of mechatronic systems. Project Based Learning (PBL) is an appropriate method for use in the learning process of…

Simultaneous vibration control and energy harvesting using actor-critic based reinforcement learning

NASA Astrophysics Data System (ADS)

Loong, Cheng Ning; Chang, C. C.; Dimitrakopoulos, Elias G.

2018-03-01

Mitigating excessive vibration of civil engineering structures using various types of devices has been a conspicuous research topic in the past few decades. Some devices, such as electromagnetic transducers, which have a capability of exerting control forces while simultaneously harvesting energy, have been proposed recently. These devices make possible a self-regenerative system that can semi-actively mitigate structural vibration without the need of external energy. Integrating mechanical, electrical components, and control algorithms, these devices open up a new research domain that needs to be addressed. In this study, the feasibility of using an actor-critic based reinforcement learning control algorithm for simultaneous vibration control and energy harvesting for a civil engineering structure is investigated. The actor-critic based reinforcement learning control algorithm is a real-time, model-free adaptive technique that can adjust the controller parameters based on observations and reward signals without knowing the system characteristics. It is suitable for the control of a partially known nonlinear system with uncertain parameters. The feasibility of implementing this algorithm on a building structure equipped with an electromagnetic damper will be investigated in this study. Issues related to the modelling of learning algorithm, initialization and convergence will be presented and discussed.

Maze learning by a hybrid brain-computer system

NASA Astrophysics Data System (ADS)

Wu, Zhaohui; Zheng, Nenggan; Zhang, Shaowu; Zheng, Xiaoxiang; Gao, Liqiang; Su, Lijuan

2016-09-01

The combination of biological and artificial intelligence is particularly driven by two major strands of research: one involves the control of mechanical, usually prosthetic, devices by conscious biological subjects, whereas the other involves the control of animal behaviour by stimulating nervous systems electrically or optically. However, to our knowledge, no study has demonstrated that spatial learning in a computer-based system can affect the learning and decision making behaviour of the biological component, namely a rat, when these two types of intelligence are wired together to form a new intelligent entity. Here, we show how rule operations conducted by computing components contribute to a novel hybrid brain-computer system, i.e., ratbots, exhibit superior learning abilities in a maze learning task, even when their vision and whisker sensation were blocked. We anticipate that our study will encourage other researchers to investigate combinations of various rule operations and other artificial intelligence algorithms with the learning and memory processes of organic brains to develop more powerful cyborg intelligence systems. Our results potentially have profound implications for a variety of applications in intelligent systems and neural rehabilitation.

Maze learning by a hybrid brain-computer system.

PubMed

Wu, Zhaohui; Zheng, Nenggan; Zhang, Shaowu; Zheng, Xiaoxiang; Gao, Liqiang; Su, Lijuan

2016-09-13

The combination of biological and artificial intelligence is particularly driven by two major strands of research: one involves the control of mechanical, usually prosthetic, devices by conscious biological subjects, whereas the other involves the control of animal behaviour by stimulating nervous systems electrically or optically. However, to our knowledge, no study has demonstrated that spatial learning in a computer-based system can affect the learning and decision making behaviour of the biological component, namely a rat, when these two types of intelligence are wired together to form a new intelligent entity. Here, we show how rule operations conducted by computing components contribute to a novel hybrid brain-computer system, i.e., ratbots, exhibit superior learning abilities in a maze learning task, even when their vision and whisker sensation were blocked. We anticipate that our study will encourage other researchers to investigate combinations of various rule operations and other artificial intelligence algorithms with the learning and memory processes of organic brains to develop more powerful cyborg intelligence systems. Our results potentially have profound implications for a variety of applications in intelligent systems and neural rehabilitation.

Maze learning by a hybrid brain-computer system

PubMed Central

Wu, Zhaohui; Zheng, Nenggan; Zhang, Shaowu; Zheng, Xiaoxiang; Gao, Liqiang; Su, Lijuan

2016-01-01

The combination of biological and artificial intelligence is particularly driven by two major strands of research: one involves the control of mechanical, usually prosthetic, devices by conscious biological subjects, whereas the other involves the control of animal behaviour by stimulating nervous systems electrically or optically. However, to our knowledge, no study has demonstrated that spatial learning in a computer-based system can affect the learning and decision making behaviour of the biological component, namely a rat, when these two types of intelligence are wired together to form a new intelligent entity. Here, we show how rule operations conducted by computing components contribute to a novel hybrid brain-computer system, i.e., ratbots, exhibit superior learning abilities in a maze learning task, even when their vision and whisker sensation were blocked. We anticipate that our study will encourage other researchers to investigate combinations of various rule operations and other artificial intelligence algorithms with the learning and memory processes of organic brains to develop more powerful cyborg intelligence systems. Our results potentially have profound implications for a variety of applications in intelligent systems and neural rehabilitation. PMID:27619326

Serendipitous Offline Learning in a Neuromorphic Robot.

PubMed

Stewart, Terrence C; Kleinhans, Ashley; Mundy, Andrew; Conradt, Jörg

2016-01-01

We demonstrate a hybrid neuromorphic learning paradigm that learns complex sensorimotor mappings based on a small set of hard-coded reflex behaviors. A mobile robot is first controlled by a basic set of reflexive hand-designed behaviors. All sensor data is provided via a spike-based silicon retina camera (eDVS), and all control is implemented via spiking neurons simulated on neuromorphic hardware (SpiNNaker). Given this control system, the robot is capable of simple obstacle avoidance and random exploration. To train the robot to perform more complex tasks, we observe the robot and find instances where the robot accidentally performs the desired action. Data recorded from the robot during these times is then used to update the neural control system, increasing the likelihood of the robot performing that task in the future, given a similar sensor state. As an example application of this general-purpose method of training, we demonstrate the robot learning to respond to novel sensory stimuli (a mirror) by turning right if it is present at an intersection, and otherwise turning left. In general, this system can learn arbitrary relations between sensory input and motor behavior.

Intelligent control of robotic arm/hand systems for the NASA EVA retriever using neural networks

NASA Technical Reports Server (NTRS)

Mclauchlan, Robert A.

1989-01-01

Adaptive/general learning algorithms using varying neural network models are considered for the intelligent control of robotic arm plus dextrous hand/manipulator systems. Results are summarized and discussed for the use of the Barto/Sutton/Anderson neuronlike, unsupervised learning controller as applied to the stabilization of an inverted pendulum on a cart system. Recommendations are made for the application of the controller and a kinematic analysis for trajectory planning to simple object retrieval (chase/approach and capture/grasp) scenarios in two dimensions.

Representing high-dimensional data to intelligent prostheses and other wearable assistive robots: A first comparison of tile coding and selective Kanerva coding.

PubMed

Travnik, Jaden B; Pilarski, Patrick M

2017-07-01

Prosthetic devices have advanced in their capabilities and in the number and type of sensors included in their design. As the space of sensorimotor data available to a conventional or machine learning prosthetic control system increases in dimensionality and complexity, it becomes increasingly important that this data be represented in a useful and computationally efficient way. Well structured sensory data allows prosthetic control systems to make informed, appropriate control decisions. In this study, we explore the impact that increased sensorimotor information has on current machine learning prosthetic control approaches. Specifically, we examine the effect that high-dimensional sensory data has on the computation time and prediction performance of a true-online temporal-difference learning prediction method as embedded within a resource-limited upper-limb prosthesis control system. We present results comparing tile coding, the dominant linear representation for real-time prosthetic machine learning, with a newly proposed modification to Kanerva coding that we call selective Kanerva coding. In addition to showing promising results for selective Kanerva coding, our results confirm potential limitations to tile coding as the number of sensory input dimensions increases. To our knowledge, this study is the first to explicitly examine representations for realtime machine learning prosthetic devices in general terms. This work therefore provides an important step towards forming an efficient prosthesis-eye view of the world, wherein prompt and accurate representations of high-dimensional data may be provided to machine learning control systems within artificial limbs and other assistive rehabilitation technologies.

Memory and cognitive control circuits in mathematical cognition and learning.

PubMed

Menon, V

2016-01-01

Numerical cognition relies on interactions within and between multiple functional brain systems, including those subserving quantity processing, working memory, declarative memory, and cognitive control. This chapter describes recent advances in our understanding of memory and control circuits in mathematical cognition and learning. The working memory system involves multiple parietal-frontal circuits which create short-term representations that allow manipulation of discrete quantities over several seconds. In contrast, hippocampal-frontal circuits underlying the declarative memory system play an important role in formation of associative memories and binding of new and old information, leading to the formation of long-term memories that allow generalization beyond individual problem attributes. The flow of information across these systems is regulated by flexible cognitive control systems which facilitate the integration and manipulation of quantity and mnemonic information. The implications of recent research for formulating a more comprehensive systems neuroscience view of the neural basis of mathematical learning and knowledge acquisition in both children and adults are discussed. © 2016 Elsevier B.V. All rights reserved.

Memory and cognitive control circuits in mathematical cognition and learning

PubMed Central

Menon, V.

2018-01-01

Numerical cognition relies on interactions within and between multiple functional brain systems, including those subserving quantity processing, working memory, declarative memory, and cognitive control. This chapter describes recent advances in our understanding of memory and control circuits in mathematical cognition and learning. The working memory system involves multiple parietal–frontal circuits which create short-term representations that allow manipulation of discrete quantities over several seconds. In contrast, hippocampal–frontal circuits underlying the declarative memory system play an important role in formation of associative memories and binding of new and old information, leading to the formation of long-term memories that allow generalization beyond individual problem attributes. The flow of information across these systems is regulated by flexible cognitive control systems which facilitate the integration and manipulation of quantity and mnemonic information. The implications of recent research for formulating a more comprehensive systems neuroscience view of the neural basis of mathematical learning and knowledge acquisition in both children and adults are discussed. PMID:27339012

Investigation of Drive-Reinforcement Learning and Application of Learning to Flight Control

DTIC Science & Technology

1993-08-01

Attachment 1 138 Reprint of: Baird, L. (1991). Learning and Adaptive Hybrid Systems for Nonlinear Control, CSDL Report T-1099, M.S. Thesis , Department of...Aircraft, CSDL Report T-1127, S.M. Thesis , Department of Aeronautics and Astronautics, M.I.T. Attachment 3 351 . iprint of: Atkins, S. (1993...Incremental Synthesis of Optimal Control Laws Using Learning Algorithms, CSDL Report T-1181, S.M. Thesis , Department of Aeronautics and Astronautics, M.I.T

A neural learning classifier system with self-adaptive constructivism for mobile robot control.

PubMed

Hurst, Jacob; Bull, Larry

2006-01-01

For artificial entities to achieve true autonomy and display complex lifelike behavior, they will need to exploit appropriate adaptable learning algorithms. In this context adaptability implies flexibility guided by the environment at any given time and an open-ended ability to learn appropriate behaviors. This article examines the use of constructivism-inspired mechanisms within a neural learning classifier system architecture that exploits parameter self-adaptation as an approach to realize such behavior. The system uses a rule structure in which each rule is represented by an artificial neural network. It is shown that appropriate internal rule complexity emerges during learning at a rate controlled by the learner and that the structure indicates underlying features of the task. Results are presented in simulated mazes before moving to a mobile robot platform.

Parent Perception of Two Eye-Gaze Control Technology Systems in Young Children with Cerebral Palsy: Pilot Study.

PubMed

Karlsson, Petra; Wallen, Margaret

2017-01-01

Eye-gaze control technology enables people with significant physical disability to access computers for communication, play, learning and environmental control. This pilot study used a multiple case study design with repeated baseline assessment and parents' evaluations to compare two eye-gaze control technology systems to identify any differences in factors such as ease of use and impact of the systems for their young children. Five children, aged 3 to 5 years, with dyskinetic cerebral palsy, and their families participated. Overall, families were satisfied with both the Tobii PCEye Go and myGaze® eye tracker, found them easy to position and use, and children learned to operate them quickly. This technology provides young children with important opportunities for learning, play, leisure, and developing communication.

Perceptual-motor skill learning in Gilles de la Tourette syndrome. Evidence for multiple procedural learning and memory systems.

PubMed

Marsh, Rachel; Alexander, Gerianne M; Packard, Mark G; Zhu, Hongtu; Peterson, Bradley S

2005-01-01

Procedural learning and memory systems likely comprise several skills that are differentially affected by various illnesses of the central nervous system, suggesting their relative functional independence and reliance on differing neural circuits. Gilles de la Tourette syndrome (GTS) is a movement disorder that involves disturbances in the structure and function of the striatum and related circuitry. Recent studies suggest that patients with GTS are impaired in performance of a probabilistic classification task that putatively involves the acquisition of stimulus-response (S-R)-based habits. Assessing the learning of perceptual-motor skills and probabilistic classification in the same samples of GTS and healthy control subjects may help to determine whether these various forms of procedural (habit) learning rely on the same or differing neuroanatomical substrates and whether those substrates are differentially affected in persons with GTS. Therefore, we assessed perceptual-motor skill learning using the pursuit-rotor and mirror tracing tasks in 50 patients with GTS and 55 control subjects who had previously been compared at learning a task of probabilistic classifications. The GTS subjects did not differ from the control subjects in performance of either the pursuit rotor or mirror-tracing tasks, although they were significantly impaired in the acquisition of a probabilistic classification task. In addition, learning on the perceptual-motor tasks was not correlated with habit learning on the classification task in either the GTS or healthy control subjects. These findings suggest that the differing forms of procedural learning are dissociable both functionally and neuroanatomically. The specific deficits in the probabilistic classification form of habit learning in persons with GTS are likely to be a consequence of disturbances in specific corticostriatal circuits, but not the same circuits that subserve the perceptual-motor form of habit learning.

Learning in tele-autonomous systems using Soar

NASA Technical Reports Server (NTRS)

Laird, John E.; Yager, Eric S.; Tuck, Christopher M.; Hucka, Michael

1989-01-01

Robo-Soar is a high-level robot arm control system implemented in Soar. Robo-Soar learns to perform simple block manipulation tasks using advice from a human. Following learning, the system is able to perform similar tasks without external guidance. It can also learn to correct its knowledge, using its own problem solving in addition to outside guidance. Robo-Soar corrects its knowledge by accepting advice about relevance of features in its domain, using a unique integration of analytic and empirical learning techniques.

Hierarchical control of procedural and declarative category-learning systems

PubMed Central

Turner, Benjamin O.; Crossley, Matthew J.; Ashby, F. Gregory

2017-01-01

Substantial evidence suggests that human category learning is governed by the interaction of multiple qualitatively distinct neural systems. In this view, procedural memory is used to learn stimulus-response associations, and declarative memory is used to apply explicit rules and test hypotheses about category membership. However, much less is known about the interaction between these systems: how is control passed between systems as they interact to influence motor resources? Here, we used fMRI to elucidate the neural correlates of switching between procedural and declarative categorization systems. We identified a key region of the cerebellum (left Crus I) whose activity was bidirectionally modulated depending on switch direction. We also identified regions of the default mode network (DMN) that were selectively connected to left Crus I during switching. We propose that the cerebellum—in coordination with the DMN—serves a critical role in passing control between procedural and declarative memory systems. PMID:28213114

Implementing Google Apps for Education as Learning Management System in Math Education

NASA Astrophysics Data System (ADS)

Widodo, S.

2017-09-01

This study aims to find the effectiveness of math education using Google Apps for Education (GAFE) as learning management system to improve mathematical communication skill primary school preservice teacher. This research used quasi-experimental approach, utilizing the control group pre-test - post-test design of two group of primary school preservice teachers at UPI Kampus Purwakarta. The result of this study showed that mathematical communication skill of primary school preservice teacher in the experiment group is better than the control group. This is because the primary school preservice teacher in the experiment group used GAFE as a tool to communicate their idea. The students can communicate their idea because they have read the learning material on the learning management system using GAFE. All in all, it can be concluded that the communication tool is very important, beside the learning material, and also the options to choose the learning model to achieve the better result.

A Project-Based Laboratory for Learning Embedded System Design with Industry Support

ERIC Educational Resources Information Center

Lee, Chyi-Shyong; Su, Juing-Huei; Lin, Kuo-En; Chang, Jia-Hao; Lin, Gu-Hong

2010-01-01

A project-based laboratory for learning embedded system design with support from industry is presented in this paper. The aim of this laboratory is to motivate students to learn the building blocks of embedded systems and practical control algorithms by constructing a line-following robot using the quadratic interpolation technique to predict the…

An open-closed-loop iterative learning control approach for nonlinear switched systems with application to freeway traffic control

NASA Astrophysics Data System (ADS)

Sun, Shu-Ting; Li, Xiao-Dong; Zhong, Ren-Xin

2017-10-01

For nonlinear switched discrete-time systems with input constraints, this paper presents an open-closed-loop iterative learning control (ILC) approach, which includes a feedforward ILC part and a feedback control part. Under a given switching rule, the mathematical induction is used to prove the convergence of ILC tracking error in each subsystem. It is demonstrated that the convergence of ILC tracking error is dependent on the feedforward control gain, but the feedback control can speed up the convergence process of ILC by a suitable selection of feedback control gain. A switched freeway traffic system is used to illustrate the effectiveness of the proposed ILC law.

A new learning strategy for the two-time-scale neural controller with its application to the tracking control of rigid arms

NASA Technical Reports Server (NTRS)

Cheng, W.; Wen, J. T.

1992-01-01

A novel fast learning rule with fast weight identification is proposed for the two-time-scale neural controller, and a two-stage learning strategy is developed for the proposed neural controller. The results of the stability analysis show that both the tracking error and the fast weight error will be uniformly bounded and converge to a bounded region which depends only on the accuracy of the slow learning if the system is sufficiently excited. The efficiency of the two-stage learning is also demonstrated by a simulation of a two-link arm.

Network congestion control algorithm based on Actor-Critic reinforcement learning model

NASA Astrophysics Data System (ADS)

Xu, Tao; Gong, Lina; Zhang, Wei; Li, Xuhong; Wang, Xia; Pan, Wenwen

2018-04-01

Aiming at the network congestion control problem, a congestion control algorithm based on Actor-Critic reinforcement learning model is designed. Through the genetic algorithm in the congestion control strategy, the network congestion problems can be better found and prevented. According to Actor-Critic reinforcement learning, the simulation experiment of network congestion control algorithm is designed. The simulation experiments verify that the AQM controller can predict the dynamic characteristics of the network system. Moreover, the learning strategy is adopted to optimize the network performance, and the dropping probability of packets is adaptively adjusted so as to improve the network performance and avoid congestion. Based on the above finding, it is concluded that the network congestion control algorithm based on Actor-Critic reinforcement learning model can effectively avoid the occurrence of TCP network congestion.

Brain computer interface learning for systems based on electrocorticography and intracortical microelectrode arrays.

PubMed

Hiremath, Shivayogi V; Chen, Weidong; Wang, Wei; Foldes, Stephen; Yang, Ying; Tyler-Kabara, Elizabeth C; Collinger, Jennifer L; Boninger, Michael L

2015-01-01

A brain-computer interface (BCI) system transforms neural activity into control signals for external devices in real time. A BCI user needs to learn to generate specific cortical activity patterns to control external devices effectively. We call this process BCI learning, and it often requires significant effort and time. Therefore, it is important to study this process and develop novel and efficient approaches to accelerate BCI learning. This article reviews major approaches that have been used for BCI learning, including computer-assisted learning, co-adaptive learning, operant conditioning, and sensory feedback. We focus on BCIs based on electrocorticography and intracortical microelectrode arrays for restoring motor function. This article also explores the possibility of brain modulation techniques in promoting BCI learning, such as electrical cortical stimulation, transcranial magnetic stimulation, and optogenetics. Furthermore, as proposed by recent BCI studies, we suggest that BCI learning is in many ways analogous to motor and cognitive skill learning, and therefore skill learning should be a useful metaphor to model BCI learning.

Amygdala subsystems and control of feeding behavior by learned cues.

PubMed

Petrovich, Gorica D; Gallagher, Michela

2003-04-01

A combination of behavioral studies and a neural systems analysis approach has proven fruitful in defining the role of the amygdala complex and associated circuits in fear conditioning. The evidence presented in this chapter suggests that this approach is also informative in the study of other adaptive functions that involve the amygdala. In this chapter we present a novel model to study learning in an appetitive context. Furthermore, we demonstrate that long-recognized connections between the amygdala and the hypothalamus play a crucial role in allowing learning to modulate feeding behavior. In the first part we describe a behavioral model for motivational learning. In this model a cue that acquires motivational properties through pairings with food delivery when an animal is hungry can override satiety and promote eating in sated rats. Next, we present evidence that a specific amygdala subsystem (basolateral area) is responsible for allowing such learned cues to control eating (override satiety and promote eating in sated rats). We also show that basolateral amygdala mediates these actions via connectivity with the lateral hypothalamus. Lastly, we present evidence that the amygdalohypothalamic system is specific for the control of eating by learned motivational cues, as it does not mediate another function that depends on intact basolateral amygdala, namely, the ability of a conditioned cue to support new learning based on its acquired value. Knowledge about neural systems through which food-associated cues specifically control feeding behavior provides a defined model for the study of learning. In addition, this model may be informative for understanding mechanisms of maladaptive aspects of learned control of eating that contribute to eating disorders and more moderate forms of overeating.

Genetic reinforcement learning through symbiotic evolution for fuzzy controller design.

PubMed

Juang, C F; Lin, J Y; Lin, C T

2000-01-01

An efficient genetic reinforcement learning algorithm for designing fuzzy controllers is proposed in this paper. The genetic algorithm (GA) adopted in this paper is based upon symbiotic evolution which, when applied to fuzzy controller design, complements the local mapping property of a fuzzy rule. Using this Symbiotic-Evolution-based Fuzzy Controller (SEFC) design method, the number of control trials, as well as consumed CPU time, are considerably reduced when compared to traditional GA-based fuzzy controller design methods and other types of genetic reinforcement learning schemes. Moreover, unlike traditional fuzzy controllers, which partition the input space into a grid, SEFC partitions the input space in a flexible way, thus creating fewer fuzzy rules. In SEFC, different types of fuzzy rules whose consequent parts are singletons, fuzzy sets, or linear equations (TSK-type fuzzy rules) are allowed. Further, the free parameters (e.g., centers and widths of membership functions) and fuzzy rules are all tuned automatically. For the TSK-type fuzzy rule especially, which put the proposed learning algorithm in use, only the significant input variables are selected to participate in the consequent of a rule. The proposed SEFC design method has been applied to different simulated control problems, including the cart-pole balancing system, a magnetic levitation system, and a water bath temperature control system. The proposed SEFC has been verified to be efficient and superior from these control problems, and from comparisons with some traditional GA-based fuzzy systems.

A new computational account of cognitive control over reinforcement-based decision-making: Modeling of a probabilistic learning task.

PubMed

Zendehrouh, Sareh

2015-11-01

Recent work on decision-making field offers an account of dual-system theory for decision-making process. This theory holds that this process is conducted by two main controllers: a goal-directed system and a habitual system. In the reinforcement learning (RL) domain, the habitual behaviors are connected with model-free methods, in which appropriate actions are learned through trial-and-error experiences. However, goal-directed behaviors are associated with model-based methods of RL, in which actions are selected using a model of the environment. Studies on cognitive control also suggest that during processes like decision-making, some cortical and subcortical structures work in concert to monitor the consequences of decisions and to adjust control according to current task demands. Here a computational model is presented based on dual system theory and cognitive control perspective of decision-making. The proposed model is used to simulate human performance on a variant of probabilistic learning task. The basic proposal is that the brain implements a dual controller, while an accompanying monitoring system detects some kinds of conflict including a hypothetical cost-conflict one. The simulation results address existing theories about two event-related potentials, namely error related negativity (ERN) and feedback related negativity (FRN), and explore the best account of them. Based on the results, some testable predictions are also presented. Copyright © 2015 Elsevier Ltd. All rights reserved.

Modelling and Optimizing Mathematics Learning in Children

ERIC Educational Resources Information Center

Käser, Tanja; Busetto, Alberto Giovanni; Solenthaler, Barbara; Baschera, Gian-Marco; Kohn, Juliane; Kucian, Karin; von Aster, Michael; Gross, Markus

2013-01-01

This study introduces a student model and control algorithm, optimizing mathematics learning in children. The adaptive system is integrated into a computer-based training system for enhancing numerical cognition aimed at children with developmental dyscalculia or difficulties in learning mathematics. The student model consists of a dynamic…

A 2D systems approach to iterative learning control for discrete linear processes with zero Markov parameters

NASA Astrophysics Data System (ADS)

Hladowski, Lukasz; Galkowski, Krzysztof; Cai, Zhonglun; Rogers, Eric; Freeman, Chris T.; Lewin, Paul L.

2011-07-01

In this article a new approach to iterative learning control for the practically relevant case of deterministic discrete linear plants with uniform rank greater than unity is developed. The analysis is undertaken in a 2D systems setting that, by using a strong form of stability for linear repetitive processes, allows simultaneous consideration of both trial-to-trial error convergence and along the trial performance, resulting in design algorithms that can be computed using linear matrix inequalities (LMIs). Finally, the control laws are experimentally verified on a gantry robot that replicates a pick and place operation commonly found in a number of applications to which iterative learning control is applicable.

Adaptive Critic Nonlinear Robust Control: A Survey.

PubMed

Wang, Ding; He, Haibo; Liu, Derong

2017-10-01

Adaptive dynamic programming (ADP) and reinforcement learning are quite relevant to each other when performing intelligent optimization. They are both regarded as promising methods involving important components of evaluation and improvement, at the background of information technology, such as artificial intelligence, big data, and deep learning. Although great progresses have been achieved and surveyed when addressing nonlinear optimal control problems, the research on robustness of ADP-based control strategies under uncertain environment has not been fully summarized. Hence, this survey reviews the recent main results of adaptive-critic-based robust control design of continuous-time nonlinear systems. The ADP-based nonlinear optimal regulation is reviewed, followed by robust stabilization of nonlinear systems with matched uncertainties, guaranteed cost control design of unmatched plants, and decentralized stabilization of interconnected systems. Additionally, further comprehensive discussions are presented, including event-based robust control design, improvement of the critic learning rule, nonlinear H ∞ control design, and several notes on future perspectives. By applying the ADP-based optimal and robust control methods to a practical power system and an overhead crane plant, two typical examples are provided to verify the effectiveness of theoretical results. Overall, this survey is beneficial to promote the development of adaptive critic control methods with robustness guarantee and the construction of higher level intelligent systems.

BlueSky Cloud Framework: An E-Learning Framework Embracing Cloud Computing

NASA Astrophysics Data System (ADS)

Dong, Bo; Zheng, Qinghua; Qiao, Mu; Shu, Jian; Yang, Jie

Currently, E-Learning has grown into a widely accepted way of learning. With the huge growth of users, services, education contents and resources, E-Learning systems are facing challenges of optimizing resource allocations, dealing with dynamic concurrency demands, handling rapid storage growth requirements and cost controlling. In this paper, an E-Learning framework based on cloud computing is presented, namely BlueSky cloud framework. Particularly, the architecture and core components of BlueSky cloud framework are introduced. In BlueSky cloud framework, physical machines are virtualized, and allocated on demand for E-Learning systems. Moreover, BlueSky cloud framework combines with traditional middleware functions (such as load balancing and data caching) to serve for E-Learning systems as a general architecture. It delivers reliable, scalable and cost-efficient services to E-Learning systems, and E-Learning organizations can establish systems through these services in a simple way. BlueSky cloud framework solves the challenges faced by E-Learning, and improves the performance, availability and scalability of E-Learning systems.

The real-time learning mechanism of the Scientific Research Associates Advanced Robotic System (SRAARS)

NASA Technical Reports Server (NTRS)

Chen, Alexander Y.

1990-01-01

Scientific research associates advanced robotic system (SRAARS) is an intelligent robotic system which has autonomous learning capability in geometric reasoning. The system is equipped with one global intelligence center (GIC) and eight local intelligence centers (LICs). It controls mainly sixteen links with fourteen active joints, which constitute two articulated arms, an extensible lower body, a vision system with two CCD cameras and a mobile base. The on-board knowledge-based system supports the learning controller with model representations of both the robot and the working environment. By consecutive verifying and planning procedures, hypothesis-and-test routines and learning-by-analogy paradigm, the system would autonomously build up its own understanding of the relationship between itself (i.e., the robot) and the focused environment for the purposes of collision avoidance, motion analysis and object manipulation. The intelligence of SRAARS presents a valuable technical advantage to implement robotic systems for space exploration and space station operations.

Effectiveness of a Learner-Directed Model for e-Learning

ERIC Educational Resources Information Center

Lee, Stella; Barker, Trevor; Kumar, Vivekanandan Suresh

2016-01-01

It is a hard task to strike a balance between extents of control a learner exercises and the amount of guidance, active or passive, afforded by the learning environment to guide, support, and motivate the learner. Adaptive systems strive to find the right balance in a spectrum that spans between self-control and system-guidance. They also concern…

Ellipsoidal fuzzy learning for smart car platoons

NASA Astrophysics Data System (ADS)

Dickerson, Julie A.; Kosko, Bart

1993-12-01

A neural-fuzzy system combined supervised and unsupervised learning to find and tune the fuzzy-rules. An additive fuzzy system approximates a function by covering its graph with fuzzy rules. A fuzzy rule patch can take the form of an ellipsoid in the input-output space. Unsupervised competitive learning found the statistics of data clusters. The covariance matrix of each synaptic quantization vector defined on ellipsoid centered at the centroid of the data cluster. Tightly clustered data gave smaller ellipsoids or more certain rules. Sparse data gave larger ellipsoids or less certain rules. Supervised learning tuned the ellipsoids to improve the approximation. The supervised neural system used gradient descent to find the ellipsoidal fuzzy patches. It locally minimized the mean-squared error of the fuzzy approximation. Hybrid ellipsoidal learning estimated the control surface for a smart car controller.

Neurocontrol and fuzzy logic: Connections and designs

NASA Technical Reports Server (NTRS)

Werbos, Paul J.

1991-01-01

Artificial neural networks (ANNs) and fuzzy logic are complementary technologies. ANNs extract information from systems to be learned or controlled, while fuzzy techniques mainly use verbal information from experts. Ideally, both sources of information should be combined. For example, one can learn rules in a hybrid fashion, and then calibrate them for better whole-system performance. ANNs offer universal approximation theorems, pedagogical advantages, very high-throughput hardware, and links to neurophysiology. Neurocontrol - the use of ANNs to directly control motors or actuators, etc. - uses five generalized designs, related to control theory, which can work on fuzzy logic systems as well as ANNs. These designs can copy what experts do instead of what they say, learn to track trajectories, generalize adaptive control, and maximize performance or minimize cost over time, even in noisy environments. Design tradeoffs and future directions are discussed throughout.

Drive Control Scheme of Electric Power Assisted Wheelchair Based on Neural Network Learning of Human Wheelchair Operation Characteristics

NASA Astrophysics Data System (ADS)

Tanohata, Naoki; Seki, Hirokazu

This paper describes a novel drive control scheme of electric power assisted wheelchairs based on neural network learning of human wheelchair operation characteristics. “Electric power assisted wheelchair” which enhances the drive force of the operator by employing electric motors is expected to be widely used as a mobility support system for elderly and disabled people. However, some handicapped people with paralysis of the muscles of one side of the body cannot maneuver the wheelchair as desired because of the difference in the right and left input force. Therefore, this study proposes a neural network learning system of such human wheelchair operation characteristics and a drive control scheme with variable distribution and assistance ratios. Some driving experiments will be performed to confirm the effectiveness of the proposed control system.

Version Control in Project-Based Learning

ERIC Educational Resources Information Center

Milentijevic, Ivan; Ciric, Vladimir; Vojinovic, Oliver

2008-01-01

This paper deals with the development of a generalized model for version control systems application as a support in a range of project-based learning methods. The model is given as UML sequence diagram and described in detail. The proposed model encompasses a wide range of different project-based learning approaches by assigning a supervisory…

Decomposed fuzzy systems and their application in direct adaptive fuzzy control.

PubMed

Hsueh, Yao-Chu; Su, Shun-Feng; Chen, Ming-Chang

2014-10-01

In this paper, a novel fuzzy structure termed as the decomposed fuzzy system (DFS) is proposed to act as the fuzzy approximator for adaptive fuzzy control systems. The proposed structure is to decompose each fuzzy variable into layers of fuzzy systems, and each layer is to characterize one traditional fuzzy set. Similar to forming fuzzy rules in traditional fuzzy systems, layers from different variables form the so-called component fuzzy systems. DFS is proposed to provide more adjustable parameters to facilitate possible adaptation in fuzzy rules, but without introducing a learning burden. It is because those component fuzzy systems are independent so that it can facilitate minimum distribution learning effects among component fuzzy systems. It can be seen from our experiments that even when the rule number increases, the learning time in terms of cycles is still almost constant. It can also be found that the function approximation capability and learning efficiency of the DFS are much better than that of the traditional fuzzy systems when employed in adaptive fuzzy control systems. Besides, in order to further reduce the computational burden, a simplified DFS is proposed in this paper to satisfy possible real time constraints required in many applications. From our simulation results, it can be seen that the simplified DFS can perform fairly with a more concise decomposition structure.

Adaptive versus Learner Control in a Multiple Intelligence Learning Environment

ERIC Educational Resources Information Center

Kelly, Declan

2008-01-01

Within the field of technology enhanced learning, adaptive educational systems offer an advanced form of learning environment that attempts to meet the needs of different students. Such systems capture and represent, for each student, various characteristics such as knowledge and traits in an individual learner model. Subsequently, using the…

Decentralized reinforcement-learning control and emergence of motion patterns

NASA Astrophysics Data System (ADS)

Svinin, Mikhail; Yamada, Kazuyaki; Okhura, Kazuhiro; Ueda, Kanji

1998-10-01

In this paper we propose a system for studying emergence of motion patterns in autonomous mobile robotic systems. The system implements an instance-based reinforcement learning control. Three spaces are of importance in formulation of the control scheme. They are the work space, the sensor space, and the action space. Important feature of our system is that all these spaces are assumed to be continuous. The core part of the system is a classifier system. Based on the sensory state space analysis, the control is decentralized and is specified at the lowest level of the control system. However, the local controllers are implicitly connected through the perceived environment information. Therefore, they constitute a dynamic environment with respect to each other. The proposed control scheme is tested under simulation for a mobile robot in a navigation task. It is shown that some patterns of global behavior--such as collision avoidance, wall-following, light-seeking--can emerge from the local controllers.

Man-Robot Symbiosis: A Framework For Cooperative Intelligence And Control

NASA Astrophysics Data System (ADS)

Parker, Lynne E.; Pin, Francois G.

1988-10-01

The man-robot symbiosis concept has the fundamental objective of bridging the gap between fully human-controlled and fully autonomous systems to achieve true man-robot cooperative control and intelligence. Such a system would allow improved speed, accuracy, and efficiency of task execution, while retaining the man in the loop for innovative reasoning and decision-making. The symbiont would have capabilities for supervised and unsupervised learning, allowing an increase of expertise in a wide task domain. This paper describes a robotic system architecture facilitating the symbiotic integration of teleoperative and automated modes of task execution. The architecture reflects a unique blend of many disciplines of artificial intelligence into a working system, including job or mission planning, dynamic task allocation, man-robot communication, automated monitoring, and machine learning. These disciplines are embodied in five major components of the symbiotic framework: the Job Planner, the Dynamic Task Allocator, the Presenter/Interpreter, the Automated Monitor, and the Learning System.

Learning classifier systems for single and multiple mobile robots in unstructured environments

NASA Astrophysics Data System (ADS)

Bay, John S.

1995-12-01

The learning classifier system (LCS) is a learning production system that generates behavioral rules via an underlying discovery mechanism. The LCS architecture operates similarly to a blackboard architecture; i.e., by posted-message communications. But in the LCS, the message board is wiped clean at every time interval, thereby requiring no persistent shared resource. In this paper, we adapt the LCS to the problem of mobile robot navigation in completely unstructured environments. We consider the model of the robot itself, including its sensor and actuator structures, to be part of this environment, in addition to the world-model that includes a goal and obstacles at unknown locations. This requires a robot to learn its own I/O characteristics in addition to solving its navigation problem, but results in a learning controller that is equally applicable, unaltered, in robots with a wide variety of kinematic structures and sensing capabilities. We show the effectiveness of this LCS-based controller through both simulation and experimental trials with a small robot. We then propose a new architecture, the Distributed Learning Classifier System (DLCS), which generalizes the message-passing behavior of the LCS from internal messages within a single agent to broadcast massages among multiple agents. This communications mode requires little bandwidth and is easily implemented with inexpensive, off-the-shelf hardware. The DLCS is shown to have potential application as a learning controller for multiple intelligent agents.

Voice over Internet Protocol (VoIP) Technology as a Global Learning Tool: Information Systems Success and Control Belief Perspectives

ERIC Educational Resources Information Center

Chen, Charlie C.; Vannoy, Sandra

2013-01-01

Voice over Internet Protocol- (VoIP) enabled online learning service providers struggling with high attrition rates and low customer loyalty issues despite VoIP's high degree of system fit for online global learning applications. Effective solutions to this prevalent problem rely on the understanding of system quality, information quality, and…

Applying Learning Analytics to Investigate Timed Release in Online Learning

ERIC Educational Resources Information Center

Martin, Florence; Whitmer, John C.

2016-01-01

Adaptive learning gives learners control of context, pace, and scope of their learning experience. This strategy can be implemented in online learning by using the "Adaptive Release" feature in learning management systems. The purpose of this study was to use learning analytics research methods to explore the extent to which the adaptive…

Experiences with establishing and implementing learning management system and computer-based test system in medical college.

PubMed

Park, Joo Hyun; Son, Ji Young; Kim, Sun

2012-09-01

The purpose of this study was to establish an e-learning system to support learning in medical education and identify solutions for improving the system. A learning management system (LMS) and computer-based test (CBT) system were established to support e-learning for medical students. A survey of 219 first- and second-grade medical students was administered. The questionnaire included 9 forced choice questions about the usability of system and 2 open-ended questions about necessary improvements to the system. The LMS consisted of a class management, class evaluation, and class attendance system. CBT consisted of a test management, item bank, and authoring tool system. The results of the survey showed a high level of satisfaction in all system usability items except for stability. Further, the advantages of the e-learning system were ensuring information accessibility, providing constant feedback, and designing an intuitive interface. Necessary improvements to the system were stability, user control, readability, and diverse device usage. Based on the findings, suggestions for developing an e-learning system to improve usability by medical students and support learning effectively are recommended.

Novel associative-memory-based self-learning neurocontrol model

NASA Astrophysics Data System (ADS)

Chen, Ke

1992-09-01

Intelligent control is an important field of AI application, which is closely related to machine learning, and the neurocontrol is a kind of intelligent control that controls actions of a physical system or a plant. Linear associative memory model is a good analytic tool for artificial neural networks. In this paper, we present a novel self-learning neurocontrol on the basis of the linear associative memory model to support intelligent control. Using our self-learning neurocontrol model, the learning process is viewed as an extension of one of J. Piaget's developmental stages. After a particular linear associative model developed by us is presented, a brief introduction to J. Piaget's cognitive theory is described as the basis of our self-learning style control. It follows that the neurocontrol model is presented, which usually includes two learning stages, viz. primary learning and high-level learning. As a demonstration of our neurocontrol model, an example is also presented with simulation techniques, called that `bird' catches an aim. The tentative experimental results show that the learning and controlling performance of this approach is surprisingly good. In conclusion, future research is pointed out to improve our self-learning neurocontrol model and explore other areas of application.

Reward-Modulated Hebbian Plasticity as Leverage for Partially Embodied Control in Compliant Robotics

PubMed Central

Burms, Jeroen; Caluwaerts, Ken; Dambre, Joni

2015-01-01

In embodied computation (or morphological computation), part of the complexity of motor control is offloaded to the body dynamics. We demonstrate that a simple Hebbian-like learning rule can be used to train systems with (partial) embodiment, and can be extended outside of the scope of traditional neural networks. To this end, we apply the learning rule to optimize the connection weights of recurrent neural networks with different topologies and for various tasks. We then apply this learning rule to a simulated compliant tensegrity robot by optimizing static feedback controllers that directly exploit the dynamics of the robot body. This leads to partially embodied controllers, i.e., hybrid controllers that naturally integrate the computations that are performed by the robot body into a neural network architecture. Our results demonstrate the universal applicability of reward-modulated Hebbian learning. Furthermore, they demonstrate the robustness of systems trained with the learning rule. This study strengthens our belief that compliant robots should or can be seen as computational units, instead of dumb hardware that needs a complex controller. This link between compliant robotics and neural networks is also the main reason for our search for simple universal learning rules for both neural networks and robotics. PMID:26347645

A global bioheat model with self-tuning optimal regulation of body temperature using Hebbian feedback covariance learning.

PubMed

Ong, M L; Ng, E Y K

2005-12-01

In the lower brain, body temperature is continually being regulated almost flawlessly despite huge fluctuations in ambient and physiological conditions that constantly threaten the well-being of the body. The underlying control problem defining thermal homeostasis is one of great enormity: Many systems and sub-systems are involved in temperature regulation and physiological processes are intrinsically complex and intertwined. Thus the defining control system has to take into account the complications of nonlinearities, system uncertainties, delayed feedback loops as well as internal and external disturbances. In this paper, we propose a self-tuning adaptive thermal controller based upon Hebbian feedback covariance learning where the system is to be regulated continually to best suit its environment. This hypothesis is supported in part by postulations of the presence of adaptive optimization behavior in biological systems of certain organisms which face limited resources vital for survival. We demonstrate the use of Hebbian feedback covariance learning as a possible self-adaptive controller in body temperature regulation. The model postulates an important role of Hebbian covariance adaptation as a means of reinforcement learning in the thermal controller. The passive system is based on a simplified 2-node core and shell representation of the body, where global responses are captured. Model predictions are consistent with observed thermoregulatory responses to conditions of exercise and rest, and heat and cold stress. An important implication of the model is that optimal physiological behaviors arising from self-tuning adaptive regulation in the thermal controller may be responsible for the departure from homeostasis in abnormal states, e.g., fever. This was previously unexplained using the conventional "set-point" control theory.

Contingency learning is reduced for high conflict stimuli.

PubMed

Whitehead, Peter S; Brewer, Gene A; Patwary, Nowed; Blais, Chris

2016-09-16

Recent theories have proposed that contingency learning occurs independent of control processes. These parallel processing accounts propose that behavioral effects originally thought to be products of control processes are in fact products solely of contingency learning. This view runs contrary to conflict-mediated Hebbian-learning models that posit control and contingency learning are parts of an interactive system. In this study we replicate the contingency learning effect and modify it to further test the veracity of the parallel processing accounts in comparison to conflict-mediated Hebbian-learning models. This is accomplished by manipulating conflict to test for an interaction, or lack thereof, between conflict and contingency learning. The results are consistent with conflict-mediated Hebbian-learning in that the addition of conflict reduces the magnitude of the contingency learning effect. Copyright © 2016 Elsevier B.V. All rights reserved.

Implementation of a Surface Electromyography-Based Upper Extremity Exoskeleton Controller Using Learning from Demonstration.

PubMed

Siu, Ho Chit; Arenas, Ana M; Sun, Tingxiao; Stirling, Leia A

2018-02-05

Upper-extremity exoskeletons have demonstrated potential as augmentative, assistive, and rehabilitative devices. Typical control of upper-extremity exoskeletons have relied on switches, force/torque sensors, and surface electromyography (sEMG), but these systems are usually reactionary, and/or rely on entirely hand-tuned parameters. sEMG-based systems may be able to provide anticipatory control, since they interface directly with muscle signals, but typically require expert placement of sensors on muscle bodies. We present an implementation of an adaptive sEMG-based exoskeleton controller that learns a mapping between muscle activation and the desired system state during interaction with a user, generating a personalized sEMG feature classifier to allow for anticipatory control. This system is robust to novice placement of sEMG sensors, as well as subdermal muscle shifts. We validate this method with 18 subjects using a thumb exoskeleton to complete a book-placement task. This learning-from-demonstration system for exoskeleton control allows for very short training times, as well as the potential for improvement in intent recognition over time, and adaptation to physiological changes in the user, such as those due to fatigue.

Implementation of a Surface Electromyography-Based Upper Extremity Exoskeleton Controller Using Learning from Demonstration

PubMed Central

Arenas, Ana M.; Sun, Tingxiao

2018-01-01

Upper-extremity exoskeletons have demonstrated potential as augmentative, assistive, and rehabilitative devices. Typical control of upper-extremity exoskeletons have relied on switches, force/torque sensors, and surface electromyography (sEMG), but these systems are usually reactionary, and/or rely on entirely hand-tuned parameters. sEMG-based systems may be able to provide anticipatory control, since they interface directly with muscle signals, but typically require expert placement of sensors on muscle bodies. We present an implementation of an adaptive sEMG-based exoskeleton controller that learns a mapping between muscle activation and the desired system state during interaction with a user, generating a personalized sEMG feature classifier to allow for anticipatory control. This system is robust to novice placement of sEMG sensors, as well as subdermal muscle shifts. We validate this method with 18 subjects using a thumb exoskeleton to complete a book-placement task. This learning-from-demonstration system for exoskeleton control allows for very short training times, as well as the potential for improvement in intent recognition over time, and adaptation to physiological changes in the user, such as those due to fatigue. PMID:29401754

Characteristics of learning voluntary control of posture in lesions of the pyramidal and nigrostriatal systems.

PubMed

Ioffe, M E; Ustinova, K I; Chernikova, L A; Luk'yanova, Yu A; Ivanova-Smolenskaya, I A; Kulikov, M A

2004-07-01

The aim of the study reported here was to investigate impairments on the learning of voluntary control of the center of pressures using visual feedback in patients with lesions of the corticospinal and nigrostriatal systems. Participants were 33 patients with Parkinson's disease and 20 patients with hemipareses due to circulatory lesions in the basin of the middle cerebral artery. Subjects stood on a stabilometric platform and used two computer games over 10 days to learn to shift the body relative to the foot to move the centre of pressures, indicated by the position of a cursor on the screen, with the target and to move the target to a specified part of the screen. The games differed in terms of the postural tasks. In one, the direction of movement of the center of pressures was not known to the subjects, and subjects learned a general strategy for posture control; the other formed a strictly defined postural coordination. Both groups of patients were found to have impairments of voluntary control of the position of the center of pressures. There were no differences between groups of patients, in terms of the severity of the initial performance deficit in the task involving shifts of the center of pressures in different directions (the general strategy for controlling the center of pressures), while learning of this task was more difficult for patients with Parkinson's disease. The initial deficit in the fine postural coordination task was more marked in patients with Parkinsonism, though learning in these patients was significantly better than in patients with hemipareses. It is suggested that the mechanisms of involvement of the nigrostriatal and corticospinal systems in learning the voluntary control of posture have elements in common as well as unique elements.

Encoder-Decoder Optimization for Brain-Computer Interfaces

PubMed Central

Merel, Josh; Pianto, Donald M.; Cunningham, John P.; Paninski, Liam

2015-01-01

Neuroprosthetic brain-computer interfaces are systems that decode neural activity into useful control signals for effectors, such as a cursor on a computer screen. It has long been recognized that both the user and decoding system can adapt to increase the accuracy of the end effector. Co-adaptation is the process whereby a user learns to control the system in conjunction with the decoder adapting to learn the user's neural patterns. We provide a mathematical framework for co-adaptation and relate co-adaptation to the joint optimization of the user's control scheme ("encoding model") and the decoding algorithm's parameters. When the assumptions of that framework are respected, co-adaptation cannot yield better performance than that obtainable by an optimal initial choice of fixed decoder, coupled with optimal user learning. For a specific case, we provide numerical methods to obtain such an optimized decoder. We demonstrate our approach in a model brain-computer interface system using an online prosthesis simulator, a simple human-in-the-loop pyschophysics setup which provides a non-invasive simulation of the BCI setting. These experiments support two claims: that users can learn encoders matched to fixed, optimal decoders and that, once learned, our approach yields expected performance advantages. PMID:26029919

Encoder-decoder optimization for brain-computer interfaces.

PubMed

Merel, Josh; Pianto, Donald M; Cunningham, John P; Paninski, Liam

2015-06-01

Neuroprosthetic brain-computer interfaces are systems that decode neural activity into useful control signals for effectors, such as a cursor on a computer screen. It has long been recognized that both the user and decoding system can adapt to increase the accuracy of the end effector. Co-adaptation is the process whereby a user learns to control the system in conjunction with the decoder adapting to learn the user's neural patterns. We provide a mathematical framework for co-adaptation and relate co-adaptation to the joint optimization of the user's control scheme ("encoding model") and the decoding algorithm's parameters. When the assumptions of that framework are respected, co-adaptation cannot yield better performance than that obtainable by an optimal initial choice of fixed decoder, coupled with optimal user learning. For a specific case, we provide numerical methods to obtain such an optimized decoder. We demonstrate our approach in a model brain-computer interface system using an online prosthesis simulator, a simple human-in-the-loop pyschophysics setup which provides a non-invasive simulation of the BCI setting. These experiments support two claims: that users can learn encoders matched to fixed, optimal decoders and that, once learned, our approach yields expected performance advantages.

Online adaptation and over-trial learning in macaque visuomotor control.

PubMed

Braun, Daniel A; Aertsen, Ad; Paz, Rony; Vaadia, Eilon; Rotter, Stefan; Mehring, Carsten

2011-01-01

When faced with unpredictable environments, the human motor system has been shown to develop optimized adaptation strategies that allow for online adaptation during the control process. Such online adaptation is to be contrasted to slower over-trial learning that corresponds to a trial-by-trial update of the movement plan. Here we investigate the interplay of both processes, i.e., online adaptation and over-trial learning, in a visuomotor experiment performed by macaques. We show that simple non-adaptive control schemes fail to perform in this task, but that a previously suggested adaptive optimal feedback control model can explain the observed behavior. We also show that over-trial learning as seen in learning and aftereffect curves can be explained by learning in a radial basis function network. Our results suggest that both the process of over-trial learning and the process of online adaptation are crucial to understand visuomotor learning.

Online Adaptation and Over-Trial Learning in Macaque Visuomotor Control

PubMed Central

Braun, Daniel A.; Aertsen, Ad; Paz, Rony; Vaadia, Eilon; Rotter, Stefan; Mehring, Carsten

2011-01-01

When faced with unpredictable environments, the human motor system has been shown to develop optimized adaptation strategies that allow for online adaptation during the control process. Such online adaptation is to be contrasted to slower over-trial learning that corresponds to a trial-by-trial update of the movement plan. Here we investigate the interplay of both processes, i.e., online adaptation and over-trial learning, in a visuomotor experiment performed by macaques. We show that simple non-adaptive control schemes fail to perform in this task, but that a previously suggested adaptive optimal feedback control model can explain the observed behavior. We also show that over-trial learning as seen in learning and aftereffect curves can be explained by learning in a radial basis function network. Our results suggest that both the process of over-trial learning and the process of online adaptation are crucial to understand visuomotor learning. PMID:21720526

Emotional Learning Based Intelligent Controllers for Rotor Flux Oriented Control of Induction Motor

NASA Astrophysics Data System (ADS)

Abdollahi, Rohollah; Farhangi, Reza; Yarahmadi, Ali

2014-08-01

This paper presents design and evaluation of a novel approach based on emotional learning to improve the speed control system of rotor flux oriented control of induction motor. The controller includes a neuro-fuzzy system with speed error and its derivative as inputs. A fuzzy critic evaluates the present situation, and provides the emotional signal (stress). The controller modifies its characteristics so that the critics stress is reduced. The comparative simulation results show that the proposed controller is more robust and hence found to be a suitable replacement of the conventional PI controller for the high performance industrial drive applications.

An iterative learning control method with application for CNC machine tools

DOE Office of Scientific and Technical Information (OSTI.GOV)

Kim, D.I.; Kim, S.

1996-01-01

A proportional, integral, and derivative (PID) type iterative learning controller is proposed for precise tracking control of industrial robots and computer numerical controller (CNC) machine tools performing repetitive tasks. The convergence of the output error by the proposed learning controller is guaranteed under a certain condition even when the system parameters are not known exactly and unknown external disturbances exist. As the proposed learning controller is repeatedly applied to the industrial robot or the CNC machine tool with the path-dependent repetitive task, the distance difference between the desired path and the actual tracked or machined path, which is one ofmore » the most significant factors in the evaluation of control performance, is progressively reduced. The experimental results demonstrate that the proposed learning controller can improve machining accuracy when the CNC machine tool performs repetitive machining tasks.« less

Aircraft adaptive learning control

NASA Technical Reports Server (NTRS)

Lee, P. S. T.; Vanlandingham, H. F.

1979-01-01

The optimal control theory of stochastic linear systems is discussed in terms of the advantages of distributed-control systems, and the control of randomly-sampled systems. An optimal solution to longitudinal control is derived and applied to the F-8 DFBW aircraft. A randomly-sampled linear process model with additive process and noise is developed.

Robust iterative learning control for multi-phase batch processes: an average dwell-time method with 2D convergence indexes

NASA Astrophysics Data System (ADS)

Wang, Limin; Shen, Yiteng; Yu, Jingxian; Li, Ping; Zhang, Ridong; Gao, Furong

2018-01-01

In order to cope with system disturbances in multi-phase batch processes with different dimensions, a hybrid robust control scheme of iterative learning control combined with feedback control is proposed in this paper. First, with a hybrid iterative learning control law designed by introducing the state error, the tracking error and the extended information, the multi-phase batch process is converted into a two-dimensional Fornasini-Marchesini (2D-FM) switched system with different dimensions. Second, a switching signal is designed using the average dwell-time method integrated with the related switching conditions to give sufficient conditions ensuring stable running for the system. Finally, the minimum running time of the subsystems and the control law gains are calculated by solving the linear matrix inequalities. Meanwhile, a compound 2D controller with robust performance is obtained, which includes a robust extended feedback control for ensuring the steady-state tracking error to converge rapidly. The application on an injection molding process displays the effectiveness and superiority of the proposed strategy.

Neural robust stabilization via event-triggering mechanism and adaptive learning technique.

PubMed

Wang, Ding; Liu, Derong

2018-06-01

The robust control synthesis of continuous-time nonlinear systems with uncertain term is investigated via event-triggering mechanism and adaptive critic learning technique. We mainly focus on combining the event-triggering mechanism with adaptive critic designs, so as to solve the nonlinear robust control problem. This can not only make better use of computation and communication resources, but also conduct controller design from the view of intelligent optimization. Through theoretical analysis, the nonlinear robust stabilization can be achieved by obtaining an event-triggered optimal control law of the nominal system with a newly defined cost function and a certain triggering condition. The adaptive critic technique is employed to facilitate the event-triggered control design, where a neural network is introduced as an approximator of the learning phase. The performance of the event-triggered robust control scheme is validated via simulation studies and comparisons. The present method extends the application domain of both event-triggered control and adaptive critic control to nonlinear systems possessing dynamical uncertainties. Copyright © 2018 Elsevier Ltd. All rights reserved.

Lessons Learned from the Crew Health Care System (CHeCS) Rack 1 Environmental Control and Life Support (ECLS) Design

NASA Technical Reports Server (NTRS)

Williams, David E.

2006-01-01

This paper will provide an overview of the International Space Station (ISS) Environmental Control and Life Support (ECLS) design of the Crew Health Care System (CHeCS) Rack 1 and it will document some of the lessons that have been learned to date for the ECLS equipment in this rack.

Event-Triggered Distributed Control of Nonlinear Interconnected Systems Using Online Reinforcement Learning With Exploration.

PubMed

Narayanan, Vignesh; Jagannathan, Sarangapani

2017-09-07

In this paper, a distributed control scheme for an interconnected system composed of uncertain input affine nonlinear subsystems with event triggered state feedback is presented by using a novel hybrid learning scheme-based approximate dynamic programming with online exploration. First, an approximate solution to the Hamilton-Jacobi-Bellman equation is generated with event sampled neural network (NN) approximation and subsequently, a near optimal control policy for each subsystem is derived. Artificial NNs are utilized as function approximators to develop a suite of identifiers and learn the dynamics of each subsystem. The NN weight tuning rules for the identifier and event-triggering condition are derived using Lyapunov stability theory. Taking into account, the effects of NN approximation of system dynamics and boot-strapping, a novel NN weight update is presented to approximate the optimal value function. Finally, a novel strategy to incorporate exploration in online control framework, using identifiers, is introduced to reduce the overall cost at the expense of additional computations during the initial online learning phase. System states and the NN weight estimation errors are regulated and local uniformly ultimately bounded results are achieved. The analytical results are substantiated using simulation studies.

Audio-Enhanced Computer Assisted Learning and Computer Controlled Audio-Instruction.

ERIC Educational Resources Information Center

Miller, K.; And Others

1983-01-01

Describes aspects of use of a microcomputer linked with a cassette recorder as a peripheral to enhance computer-assisted learning (CAL) and a microcomputer-controlled tape recorder linked with a microfiche reader in a commercially available teaching system. References and a listing of control programs are appended. (EJS)

Synergetic motor control paradigm for optimizing energy efficiency of multijoint reaching via tacit learning

PubMed Central

Hayashibe, Mitsuhiro; Shimoda, Shingo

2014-01-01

A human motor system can improve its behavior toward optimal movement. The skeletal system has more degrees of freedom than the task dimensions, which incurs an ill-posed problem. The multijoint system involves complex interaction torques between joints. To produce optimal motion in terms of energy consumption, the so-called cost function based optimization has been commonly used in previous works.Even if it is a fact that an optimal motor pattern is employed phenomenologically, there is no evidence that shows the existence of a physiological process that is similar to such a mathematical optimization in our central nervous system.In this study, we aim to find a more primitive computational mechanism with a modular configuration to realize adaptability and optimality without prior knowledge of system dynamics.We propose a novel motor control paradigm based on tacit learning with task space feedback. The motor command accumulation during repetitive environmental interactions, play a major role in the learning process. It is applied to a vertical cyclic reaching which involves complex interaction torques.We evaluated whether the proposed paradigm can learn how to optimize solutions with a 3-joint, planar biomechanical model. The results demonstrate that the proposed method was valid for acquiring motor synergy and resulted in energy efficient solutions for different load conditions. The case in feedback control is largely affected by the interaction torques. In contrast, the trajectory is corrected over time with tacit learning toward optimal solutions.Energy efficient solutions were obtained by the emergence of motor synergy. During learning, the contribution from feedforward controller is augmented and the one from the feedback controller is significantly minimized down to 12% for no load at hand, 16% for a 0.5 kg load condition.The proposed paradigm could provide an optimization process in redundant system with dynamic-model-free and cost-function-free approach. PMID:24616695

Synergetic motor control paradigm for optimizing energy efficiency of multijoint reaching via tacit learning.

PubMed

Hayashibe, Mitsuhiro; Shimoda, Shingo

2014-01-01

A human motor system can improve its behavior toward optimal movement. The skeletal system has more degrees of freedom than the task dimensions, which incurs an ill-posed problem. The multijoint system involves complex interaction torques between joints. To produce optimal motion in terms of energy consumption, the so-called cost function based optimization has been commonly used in previous works.Even if it is a fact that an optimal motor pattern is employed phenomenologically, there is no evidence that shows the existence of a physiological process that is similar to such a mathematical optimization in our central nervous system.In this study, we aim to find a more primitive computational mechanism with a modular configuration to realize adaptability and optimality without prior knowledge of system dynamics.We propose a novel motor control paradigm based on tacit learning with task space feedback. The motor command accumulation during repetitive environmental interactions, play a major role in the learning process. It is applied to a vertical cyclic reaching which involves complex interaction torques.We evaluated whether the proposed paradigm can learn how to optimize solutions with a 3-joint, planar biomechanical model. The results demonstrate that the proposed method was valid for acquiring motor synergy and resulted in energy efficient solutions for different load conditions. The case in feedback control is largely affected by the interaction torques. In contrast, the trajectory is corrected over time with tacit learning toward optimal solutions.Energy efficient solutions were obtained by the emergence of motor synergy. During learning, the contribution from feedforward controller is augmented and the one from the feedback controller is significantly minimized down to 12% for no load at hand, 16% for a 0.5 kg load condition.The proposed paradigm could provide an optimization process in redundant system with dynamic-model-free and cost-function-free approach.

Development and Evaluation of Mechatronics Learning System in a Web-Based Environment

ERIC Educational Resources Information Center

Shyr, Wen-Jye

2011-01-01

The development of remote laboratory suitable for the reinforcement of undergraduate level teaching of mechatronics is important. For the reason, a Web-based mechatronics learning system, called the RECOLAB (REmote COntrol LABoratory), for remote learning in engineering education has been developed in this study. The web-based environment is an…

Lessons learned on the Ground Test Accelerator control system

DOE Office of Scientific and Technical Information (OSTI.GOV)

Kozubal, A.J.; Weiss, R.E.

1994-09-01

When we initiated the control system design for the Ground Test Accelerator (GTA), we envisioned a system that would be flexible enough to handle the changing requirements of an experimental project. This control system would use a developers` toolkit to reduce the cost and time to develop applications for GTA, and through the use of open standards, the system would accommodate unforeseen requirements as they arose. Furthermore, we would attempt to demonstrate on GTA a level of automation far beyond that achieved by existing accelerator control systems. How well did we achieve these goals? What were the stumbling blocks tomore » deploying the control system, and what assumptions did we make about requirements that turned out to be incorrect? In this paper we look at the process of developing a control system that evolved into what is now the ``Experimental Physics and Industrial Control System`` (EPICS). Also, we assess the impact of this system on the GTA project, as well as the impact of GTA on EPICS. The lessons learned on GTA will be valuable for future projects.« less

Combining metric episodes with semantic event concepts within the Symbolic and Sub-Symbolic Robotics Intelligence Control System (SS-RICS)

NASA Astrophysics Data System (ADS)

Kelley, Troy D.; McGhee, S.

2013-05-01

This paper describes the ongoing development of a robotic control architecture that inspired by computational cognitive architectures from the discipline of cognitive psychology. The Symbolic and Sub-Symbolic Robotics Intelligence Control System (SS-RICS) combines symbolic and sub-symbolic representations of knowledge into a unified control architecture. The new architecture leverages previous work in cognitive architectures, specifically the development of the Adaptive Character of Thought-Rational (ACT-R) and Soar. This paper details current work on learning from episodes or events. The use of episodic memory as a learning mechanism has, until recently, been largely ignored by computational cognitive architectures. This paper details work on metric level episodic memory streams and methods for translating episodes into abstract schemas. The presentation will include research on learning through novelty and self generated feedback mechanisms for autonomous systems.

Grounding cognitive control in associative learning.

PubMed

Abrahamse, Elger; Braem, Senne; Notebaert, Wim; Verguts, Tom

2016-07-01

Cognitive control covers a broad range of cognitive functions, but its research and theories typically remain tied to a single domain. Here we outline and review an associative learning perspective on cognitive control in which control emerges from associative networks containing perceptual, motor, and goal representations. Our review identifies 3 trending research themes that are shared between the domains of conflict adaptation, task switching, response inhibition, and attentional control: Cognitive control is context-specific, can operate in the absence of awareness, and is modulated by reward. As these research themes can be envisaged as key characteristics of learning, we propose that their joint emergence across domains is not coincidental but rather reflects a (latent) growth of interest in learning-based control. Associative learning has the potential for providing broad-scaled integration to cognitive control theory, and offers a promising avenue for understanding cognitive control as a self-regulating system without postulating an ill-defined set of homunculi. We discuss novel predictions, theoretical implications, and immediate challenges that accompany an associative learning perspective on cognitive control. (PsycINFO Database Record (c) 2016 APA, all rights reserved).

A proof-of-principle simulation for closed-loop control based on preexisting experimental thalamic DBS-enhanced instrumental learning.

PubMed

Wang, Ching-Fu; Yang, Shih-Hung; Lin, Sheng-Huang; Chen, Po-Chuan; Lo, Yu-Chun; Pan, Han-Chi; Lai, Hsin-Yi; Liao, Lun-De; Lin, Hui-Ching; Chen, Hsu-Yan; Huang, Wei-Chen; Huang, Wun-Jhu; Chen, You-Yin

Deep brain stimulation (DBS) has been applied as an effective therapy for treating Parkinson's disease or essential tremor. Several open-loop DBS control strategies have been developed for clinical experiments, but they are limited by short battery life and inefficient therapy. Therefore, many closed-loop DBS control systems have been designed to tackle these problems by automatically adjusting the stimulation parameters via feedback from neural signals, which has been reported to reduce the power consumption. However, when the association between the biomarkers of the model and stimulation is unclear, it is difficult to develop an optimal control scheme for other DBS applications, i.e., DBS-enhanced instrumental learning. Furthermore, few studies have investigated the effect of closed-loop DBS control for cognition function, such as instrumental skill learning, and have been implemented in simulation environments. In this paper, we proposed a proof-of-principle design for a closed-loop DBS system, cognitive-enhancing DBS (ceDBS), which enhanced skill learning based on in vivo experimental data. The ceDBS acquired local field potential (LFP) signal from the thalamic central lateral (CL) nuclei of animals through a neural signal processing system. A strong coupling of the theta oscillation (4-7 Hz) and the learning period was found in the water reward-related lever-pressing learning task. Therefore, the theta-band power ratio, which was the averaged theta band to averaged total band (1-55 Hz) power ratio, could be used as a physiological marker for enhancement of instrumental skill learning. The on-line extraction of the theta-band power ratio was implemented on a field-programmable gate array (FPGA). An autoregressive with exogenous inputs (ARX)-based predictor was designed to construct a CL-thalamic DBS model and forecast the future physiological marker according to the past physiological marker and applied DBS. The prediction could further assist the design of a closed-loop DBS controller. A DBS controller based on a fuzzy expert system was devised to automatically control DBS according to the predicted physiological marker via a set of rules. The simulated experimental results demonstrate that the ceDBS based on the closed-loop control architecture not only reduced power consumption using the predictive physiological marker, but also achieved a desired level of physiological marker through the DBS controller. Copyright © 2017 Elsevier Inc. All rights reserved.

A neurocomputational theory of how explicit learning bootstraps early procedural learning.

PubMed

Paul, Erick J; Ashby, F Gregory

2013-01-01

It is widely accepted that human learning and memory is mediated by multiple memory systems that are each best suited to different requirements and demands. Within the domain of categorization, at least two systems are thought to facilitate learning: an explicit (declarative) system depending largely on the prefrontal cortex, and a procedural (non-declarative) system depending on the basal ganglia. Substantial evidence suggests that each system is optimally suited to learn particular categorization tasks. However, it remains unknown precisely how these systems interact to produce optimal learning and behavior. In order to investigate this issue, the present research evaluated the progression of learning through simulation of categorization tasks using COVIS, a well-known model of human category learning that includes both explicit and procedural learning systems. Specifically, the model's parameter space was thoroughly explored in procedurally learned categorization tasks across a variety of conditions and architectures to identify plausible interaction architectures. The simulation results support the hypothesis that one-way interaction between the systems occurs such that the explicit system "bootstraps" learning early on in the procedural system. Thus, the procedural system initially learns a suboptimal strategy employed by the explicit system and later refines its strategy. This bootstrapping could be from cortical-striatal projections that originate in premotor or motor regions of cortex, or possibly by the explicit system's control of motor responses through basal ganglia-mediated loops.

Composite Intelligent Learning Control of Strict-Feedback Systems With Disturbance.

PubMed

Xu, Bin; Sun, Fuchun

2018-02-01

This paper addresses the dynamic surface control of uncertain nonlinear systems on the basis of composite intelligent learning and disturbance observer in presence of unknown system nonlinearity and time-varying disturbance. The serial-parallel estimation model with intelligent approximation and disturbance estimation is built to obtain the prediction error and in this way the composite law for weights updating is constructed. The nonlinear disturbance observer is developed using intelligent approximation information while the disturbance estimation is guaranteed to converge to a bounded compact set. The highlight is that different from previous work directly toward asymptotic stability, the transparency of the intelligent approximation and disturbance estimation is included in the control scheme. The uniformly ultimate boundedness stability is analyzed via Lyapunov method. Through simulation verification, the composite intelligent learning with disturbance observer can efficiently estimate the effect caused by system nonlinearity and disturbance while the proposed approach obtains better performance with higher accuracy.

Neuromorphic learning of continuous-valued mappings in the presence of noise: Application to real-time adaptive control

NASA Technical Reports Server (NTRS)

Troudet, Terry; Merrill, Walter C.

1989-01-01

The ability of feed-forward neural net architectures to learn continuous-valued mappings in the presence of noise is demonstrated in relation to parameter identification and real-time adaptive control applications. Factors and parameters influencing the learning performance of such nets in the presence of noise are identified. Their effects are discussed through a computer simulation of the Back-Error-Propagation algorithm by taking the example of the cart-pole system controlled by a nonlinear control law. Adequate sampling of the state space is found to be essential for canceling the effect of the statistical fluctuations and allowing learning to take place.

Designing Interactive Learning Systems.

ERIC Educational Resources Information Center

Barker, Philip

1990-01-01

Describes multimedia, computer-based interactive learning systems that support various forms of individualized study. Highlights include design models; user interfaces; design guidelines; media utilization paradigms, including hypermedia and learner-controlled models; metaphors and myths; authoring tools; optical media; workstations; four case…

Zebra finch mates use their forebrain song system in unlearned call communication.

PubMed

Ter Maat, Andries; Trost, Lisa; Sagunsky, Hannes; Seltmann, Susanne; Gahr, Manfred

2014-01-01

Unlearned calls are produced by all birds whereas learned songs are only found in three avian taxa, most notably in songbirds. The neural basis for song learning and production is formed by interconnected song nuclei: the song control system. In addition to song, zebra finches produce large numbers of soft, unlearned calls, among which "stack" calls are uttered frequently. To determine unequivocally the calls produced by each member of a group, we mounted miniature wireless microphones on each zebra finch. We find that group living paired males and females communicate using bilateral stack calling. To investigate the role of the song control system in call-based male female communication, we recorded the electrical activity in a premotor nucleus of the song control system in freely behaving male birds. The unique combination of acoustic monitoring together with wireless brain recording of individual zebra finches in groups shows that the neuronal activity of the song system correlates with the production of unlearned stack calls. The results suggest that the song system evolved from a brain circuit controlling simple unlearned calls to a system capable of producing acoustically rich, learned vocalizations.

Zebra Finch Mates Use Their Forebrain Song System in Unlearned Call Communication

PubMed Central

Ter Maat, Andries; Trost, Lisa; Sagunsky, Hannes; Seltmann, Susanne; Gahr, Manfred

2014-01-01

Unlearned calls are produced by all birds whereas learned songs are only found in three avian taxa, most notably in songbirds. The neural basis for song learning and production is formed by interconnected song nuclei: the song control system. In addition to song, zebra finches produce large numbers of soft, unlearned calls, among which “stack” calls are uttered frequently. To determine unequivocally the calls produced by each member of a group, we mounted miniature wireless microphones on each zebra finch. We find that group living paired males and females communicate using bilateral stack calling. To investigate the role of the song control system in call-based male female communication, we recorded the electrical activity in a premotor nucleus of the song control system in freely behaving male birds. The unique combination of acoustic monitoring together with wireless brain recording of individual zebra finches in groups shows that the neuronal activity of the song system correlates with the production of unlearned stack calls. The results suggest that the song system evolved from a brain circuit controlling simple unlearned calls to a system capable of producing acoustically rich, learned vocalizations. PMID:25313846

Event-Triggered Distributed Approximate Optimal State and Output Control of Affine Nonlinear Interconnected Systems.

PubMed

Narayanan, Vignesh; Jagannathan, Sarangapani

2017-06-08

This paper presents an approximate optimal distributed control scheme for a known interconnected system composed of input affine nonlinear subsystems using event-triggered state and output feedback via a novel hybrid learning scheme. First, the cost function for the overall system is redefined as the sum of cost functions of individual subsystems. A distributed optimal control policy for the interconnected system is developed using the optimal value function of each subsystem. To generate the optimal control policy, forward-in-time, neural networks are employed to reconstruct the unknown optimal value function at each subsystem online. In order to retain the advantages of event-triggered feedback for an adaptive optimal controller, a novel hybrid learning scheme is proposed to reduce the convergence time for the learning algorithm. The development is based on the observation that, in the event-triggered feedback, the sampling instants are dynamic and results in variable interevent time. To relax the requirement of entire state measurements, an extended nonlinear observer is designed at each subsystem to recover the system internal states from the measurable feedback. Using a Lyapunov-based analysis, it is demonstrated that the system states and the observer errors remain locally uniformly ultimately bounded and the control policy converges to a neighborhood of the optimal policy. Simulation results are presented to demonstrate the performance of the developed controller.

Effect of audience response system technology on learning outcomes in health students and professionals: an updated systematic review.

PubMed

Atlantis, Evan; Cheema, Birinder S

2015-03-01

: Audience response system (ARS) technology is a recent innovation that is increasingly being used by health educators to improve learning outcomes. Equivocal results from previous systematic review research provide weak support for the use of ARS for improving learning outcomes at both short and long terms. This review sought to update and critically review the body of controlled experimental evidence on the use of ARS technology on learning outcomes in health students and professionals. This review searched using all identified keywords both electronic databases (CINAHL, Embase, ERIC, Medline, Science Direct, Scopus, and Web of Science) and reference lists of retrieved articles to find relevant published studies for review, from 2010 to April 2014. A descriptive synthesis of important study characteristics and effect estimates for learning outcomes was done. Three controlled trials in 321 participants from the United States were included for review. ARS knowledge retention scores were lower than the control group in one study, higher than control group provided that immediate feedback was given about each question in one study, and equivalent between intervention and control groups in another study. There is an absence of good quality evidence on effectiveness of ARS technologies for improving learning outcomes in health students and professionals.

Learning and Memory Impairments in Patients with Minimal Hepatic Encephalopathy are Associated with Structural and Functional Connectivity Alterations in Hippocampus.

PubMed

García-García, Raquel; Cruz-Gómez, Álvaro Javier; Urios, Amparo; Mangas-Losada, Alba; Forn, Cristina; Escudero-García, Desamparados; Kosenko, Elena; Torregrosa, Isidro; Tosca, Joan; Giner-Durán, Remedios; Serra, Miguel Angel; Avila, César; Belloch, Vicente; Felipo, Vicente; Montoliu, Carmina

2018-06-25

Patients with minimal hepatic encephalopathy (MHE) show mild cognitive impairment associated with alterations in attentional and executive networks. There are no studies evaluating the relationship between memory in MHE and structural and functional connectivity (FC) changes in the hippocampal system. This study aimed to evaluate verbal learning and long-term memory in cirrhotic patients with (C-MHE) and without MHE (C-NMHE) and healthy controls. We assessed the relationship between alterations in memory and the structural integrity and FC of the hippocampal system. C-MHE patients showed impairments in learning, long-term memory, and recognition, compared to C-NMHE patients and controls. Cirrhotic patients showed reduced fimbria volume compared to controls. Larger volumes in hippocampus subfields were related to better memory performance in C-NMHE patients and controls. C-MHE patients presented lower FC between the L-presubiculum and L-precuneus than C-NMHE patients. Compared to controls, C-MHE patients had reduced FC between L-presubiculum and subiculum seeds and bilateral precuneus, which correlated with cognitive impairment and memory performance. Alterations in the FC of the hippocampal system could contribute to learning and long-term memory impairments in C-MHE patients. This study demonstrates the association between alterations in learning and long-term memory and structural and FC disturbances in hippocampal structures in cirrhotic patients.

Action Control, L2 Motivational Self System, and Motivated Learning Behavior in a Foreign Language Learning Context

ERIC Educational Resources Information Center

Khany, Reza; Amiri, Majid

2018-01-01

Theoretical developments in second or foreign language motivation research have led to a better understanding of the convoluted nature of motivation in the process of language acquisition. Among these theories, action control theory has recently shown a good deal of explanatory power in second language learning contexts and in the presence of…

Nicotine facilitates memory consolidation in perceptual learning.

PubMed

Beer, Anton L; Vartak, Devavrat; Greenlee, Mark W

2013-01-01

Perceptual learning is a special type of non-declarative learning that involves experience-dependent plasticity in sensory cortices. The cholinergic system is known to modulate declarative learning. In particular, reduced levels or efficacy of the neurotransmitter acetylcholine were found to facilitate declarative memory consolidation. However, little is known about the role of the cholinergic system in memory consolidation of non-declarative learning. Here we compared two groups of non-smoking men who learned a visual texture discrimination task (TDT). One group received chewing tobacco containing nicotine for 1 h directly following the TDT training. The other group received a similar tasting control substance without nicotine. Electroencephalographic recordings during substance consumption showed reduced alpha activity and P300 latencies in the nicotine group compared to the control group. When re-tested on the TDT the following day, both groups responded more accurately and more rapidly than during training. These improvements were specific to the retinal location and orientation of the texture elements of the TDT suggesting that learning involved early visual cortex. A group comparison showed that learning effects were more pronounced in the nicotine group than in the control group. These findings suggest that oral consumption of nicotine enhances the efficacy of nicotinic acetylcholine receptors. Our findings further suggest that enhanced efficacy of the cholinergic system facilitates memory consolidation in perceptual learning (and possibly other types of non-declarative learning). In that regard acetylcholine seems to affect consolidation processes in perceptual learning in a different manner than in declarative learning. Alternatively, our findings might reflect dose-dependent cholinergic modulation of memory consolidation. This article is part of a Special Issue entitled 'Cognitive Enhancers'. Copyright © 2012 Elsevier Ltd. All rights reserved.

Development of an E-learning System for the Endoscopic Diagnosis of Early Gastric Cancer: An International Multicenter Randomized Controlled Trial.

PubMed

Yao, K; Uedo, N; Muto, M; Ishikawa, H; Cardona, H J; Filho, E C Castro; Pittayanon, R; Olano, C; Yao, F; Parra-Blanco, A; Ho, S H; Avendano, A G; Piscoya, A; Fedorov, E; Bialek, A P; Mitrakov, A; Caro, L; Gonen, C; Dolwani, S; Farca, A; Cuaresma, L F; Bonilla, J J; Kasetsermwiriya, W; Ragunath, K; Kim, S E; Marini, M; Li, H; Cimmino, D G; Piskorz, M M; Iacopini, F; So, J B; Yamazaki, K; Kim, G H; Ang, T L; Milhomem-Cardoso, D M; Waldbaum, C A; Carvajal, W A Piedra; Hayward, C M; Singh, R; Banerjee, R; Anagnostopoulos, G K; Takahashi, Y

2016-07-01

In many countries, gastric cancer is not diagnosed until an advanced stage. An Internet-based e-learning system to improve the ability of endoscopists to diagnose gastric cancer at an early stage was developed and was evaluated for its effectiveness. The study was designed as a randomized controlled trial. After receiving a pre-test, participants were randomly allocated to either an e-learning or non-e-learning group. Only those in the e-learning group gained access to the e-learning system. Two months after the pre-test, both groups received a post-test. The primary endpoint was the difference between the two groups regarding the rate of improvement of their test results. 515 endoscopists from 35 countries were assessed for eligibility, and 332 were enrolled in the study, with 166 allocated to each group. Of these, 151 participants in the e-learning group and 144 in the non-e-learning group were included in the analysis. The mean improvement rate (standard deviation) in the e-learning and non-e-learning groups was 1·24 (0·26) and 1·00 (0·16), respectively (P<0·001). This global study clearly demonstrated the efficacy of an e-learning system to expand knowledge and provide invaluable experience regarding the endoscopic detection of early gastric cancer (R000012039). Copyright © 2016 The Authors. Published by Elsevier B.V. All rights reserved.

Relations of Cognitive and Motivational Variables with Students' Human Circulatory System Achievement in Traditional and Learning Cycle Classrooms

ERIC Educational Resources Information Center

Sadi, Özlem; Çakiroglu, Jale

2014-01-01

This study is aimed at investigating the relationships among students' relevant prior knowledge, meaningful learning orientation, reasoning ability, self-efficacy, locus of control, attitudes toward biology and achievement with the human circulatory system (HCS) using the learning cycle (LC) and the traditional classroom setting. The study was…

A Computer-Assisted Learning Model Based on the Digital Game Exponential Reward System

ERIC Educational Resources Information Center

Moon, Man-Ki; Jahng, Surng-Gahb; Kim, Tae-Yong

2011-01-01

The aim of this research was to construct a motivational model which would stimulate voluntary and proactive learning using digital game methods offering players more freedom and control. The theoretical framework of this research lays the foundation for a pedagogical learning model based on digital games. We analyzed the game reward system, which…

Some problems with the design of self-learning management systems

NASA Technical Reports Server (NTRS)

Flikop, Ziny

1992-01-01

In this paper some problems in the design of management systems for complex objects are discussed. Considering the absence of adequate models and the fact that human expertise in the management of non-stationary objects becomes obsolete quickly, the use of self-learning together with a two-step optimization of on-line control rules is suggested. To prepare for the object analysis, a set of definitions has been proposed. Traditional and fuzzy sets approaches are used in the analysis. To decrease the reaction time of the control system, we propose the development of control rules without feedback.

Application of machine learning and expert systems to Statistical Process Control (SPC) chart interpretation

NASA Technical Reports Server (NTRS)

Shewhart, Mark

1991-01-01

Statistical Process Control (SPC) charts are one of several tools used in quality control. Other tools include flow charts, histograms, cause and effect diagrams, check sheets, Pareto diagrams, graphs, and scatter diagrams. A control chart is simply a graph which indicates process variation over time. The purpose of drawing a control chart is to detect any changes in the process signalled by abnormal points or patterns on the graph. The Artificial Intelligence Support Center (AISC) of the Acquisition Logistics Division has developed a hybrid machine learning expert system prototype which automates the process of constructing and interpreting control charts.

Towards autonomous neuroprosthetic control using Hebbian reinforcement learning.

PubMed

Mahmoudi, Babak; Pohlmeyer, Eric A; Prins, Noeline W; Geng, Shijia; Sanchez, Justin C

2013-12-01

Our goal was to design an adaptive neuroprosthetic controller that could learn the mapping from neural states to prosthetic actions and automatically adjust adaptation using only a binary evaluative feedback as a measure of desirability/undesirability of performance. Hebbian reinforcement learning (HRL) in a connectionist network was used for the design of the adaptive controller. The method combines the efficiency of supervised learning with the generality of reinforcement learning. The convergence properties of this approach were studied using both closed-loop control simulations and open-loop simulations that used primate neural data from robot-assisted reaching tasks. The HRL controller was able to perform classification and regression tasks using its episodic and sequential learning modes, respectively. In our experiments, the HRL controller quickly achieved convergence to an effective control policy, followed by robust performance. The controller also automatically stopped adapting the parameters after converging to a satisfactory control policy. Additionally, when the input neural vector was reorganized, the controller resumed adaptation to maintain performance. By estimating an evaluative feedback directly from the user, the HRL control algorithm may provide an efficient method for autonomous adaptation of neuroprosthetic systems. This method may enable the user to teach the controller the desired behavior using only a simple feedback signal.

Barrier Function-Based Neural Adaptive Control With Locally Weighted Learning and Finite Neuron Self-Growing Strategy.

PubMed

Jia, Zi-Jun; Song, Yong-Duan

2017-06-01

This paper presents a new approach to construct neural adaptive control for uncertain nonaffine systems. By integrating locally weighted learning with barrier Lyapunov function (BLF), a novel control design method is presented to systematically address the two critical issues in neural network (NN) control field: one is how to fulfill the compact set precondition for NN approximation, and the other is how to use varying rather than a fixed NN structure to improve the functionality of NN control. A BLF is exploited to ensure the NN inputs to remain bounded during the entire system operation. To account for system nonlinearities, a neuron self-growing strategy is proposed to guide the process for adding new neurons to the system, resulting in a self-adjustable NN structure for better learning capabilities. It is shown that the number of neurons needed to accomplish the control task is finite, and better performance can be obtained with less number of neurons as compared with traditional methods. The salient feature of the proposed method also lies in the continuity of the control action everywhere. Furthermore, the resulting control action is smooth almost everywhere except for a few time instants at which new neurons are added. Numerical example illustrates the effectiveness of the proposed approach.

Toward experimental validation of a model for human sensorimotor learning and control in teleoperation

NASA Astrophysics Data System (ADS)

Roth, Eatai; Howell, Darrin; Beckwith, Cydney; Burden, Samuel A.

2017-05-01

Humans, interacting with cyber-physical systems (CPS), formulate beliefs about the system's dynamics. It is natural to expect that human operators, tasked with teleoperation, use these beliefs to control the remote robot. For tracking tasks in the resulting human-cyber-physical system (HCPS), theory suggests that human operators can achieve exponential tracking (in stable systems) without state estimation provided they possess an accurate model of the system's dynamics. This internalized inverse model, however, renders a portion of the system state unobservable to the human operator—the zero dynamics. Prior work shows humans can track through observable linear dynamics, thus we focus on nonlinear dynamics rendered unobservable through tracking control. We propose experiments to assess the human operator's ability to learn and invert such models, and distinguish this behavior from that achieved by pure feedback control.

Verbal implicit sequence learning in persons who stutter and persons with Parkinson's disease.

PubMed

Smits-Bandstra, Sarah; Gracco, Vincent

2013-01-01

The authors investigated the integrity of implicit learning systems in 14 persons with Parkinson's disease (PPD), 14 persons who stutter (PWS), and 14 control participants. In a 120-min session participants completed a verbal serial reaction time task, naming aloud 4 syllables in response to 4 visual stimuli. Unbeknownst to participants, the syllables formed a repeating 8-item sequence. PWS and PPD demonstrated slower reaction times for early but not late learning trials relative to controls reflecting delays but not deficiencies in general learning. PPD also demonstrated less accuracy in general learning relative to controls. All groups demonstrated similar limited explicit sequence knowledge. Both PWS and PPD demonstrated significantly less implicit sequence learning relative to controls, suggesting that stuttering may be associated with compromised functional integrity of the cortico-striato-thalamo-cortical loop.

Computer-aided auscultation learning system for nursing technique instruction.

PubMed

Hou, Chun-Ju; Chen, Yen-Ting; Hu, Ling-Chen; Chuang, Chih-Chieh; Chiu, Yu-Hsien; Tsai, Ming-Shih

2008-01-01

Pulmonary auscultation is a physical assessment skill learned by nursing students for examining the respiratory system. Generally, a sound simulator equipped mannequin is used to group teach auscultation techniques via classroom demonstration. However, nursing students cannot readily duplicate this learning environment for self-study. The advancement of electronic and digital signal processing technologies facilitates simulating this learning environment. This study aims to develop a computer-aided auscultation learning system for assisting teachers and nursing students in auscultation teaching and learning. This system provides teachers with signal recording and processing of lung sounds and immediate playback of lung sounds for students. A graphical user interface allows teachers to control the measuring device, draw lung sound waveforms, highlight lung sound segments of interest, and include descriptive text. Effects on learning lung sound auscultation were evaluated for verifying the feasibility of the system. Fifteen nursing students voluntarily participated in the repeated experiment. The results of a paired t test showed that auscultative abilities of the students were significantly improved by using the computer-aided auscultation learning system.

Working Memory Contributions to Reinforcement Learning Impairments in Schizophrenia

PubMed Central

Brown, Jaime K.; Gold, James M.; Waltz, James A.; Frank, Michael J.

2014-01-01

Previous research has shown that patients with schizophrenia are impaired in reinforcement learning tasks. However, behavioral learning curves in such tasks originate from the interaction of multiple neural processes, including the basal ganglia- and dopamine-dependent reinforcement learning (RL) system, but also prefrontal cortex-dependent cognitive strategies involving working memory (WM). Thus, it is unclear which specific system induces impairments in schizophrenia. We recently developed a task and computational model allowing us to separately assess the roles of RL (slow, cumulative learning) mechanisms versus WM (fast but capacity-limited) mechanisms in healthy adult human subjects. Here, we used this task to assess patients' specific sources of impairments in learning. In 15 separate blocks, subjects learned to pick one of three actions for stimuli. The number of stimuli to learn in each block varied from two to six, allowing us to separate influences of capacity-limited WM from the incremental RL system. As expected, both patients (n = 49) and healthy controls (n = 36) showed effects of set size and delay between stimulus repetitions, confirming the presence of working memory effects. Patients performed significantly worse than controls overall, but computational model fits and behavioral analyses indicate that these deficits could be entirely accounted for by changes in WM parameters (capacity and reliability), whereas RL processes were spared. These results suggest that the working memory system contributes strongly to learning impairments in schizophrenia. PMID:25297101

Performance of children with developmental dyslexia on high and low topological entropy artificial grammar learning task.

PubMed

Katan, Pesia; Kahta, Shani; Sasson, Ayelet; Schiff, Rachel

2017-07-01

Graph complexity as measured by topological entropy has been previously shown to affect performance on artificial grammar learning tasks among typically developing children. The aim of this study was to examine the effect of graph complexity on implicit sequential learning among children with developmental dyslexia. Our goal was to determine whether children's performance depends on the complexity level of the grammar system learned. We conducted two artificial grammar learning experiments that compared performance of children with developmental dyslexia with that of age- and reading level-matched controls. Experiment 1 was a high topological entropy artificial grammar learning task that aimed to establish implicit learning phenomena in children with developmental dyslexia using previously published experimental conditions. Experiment 2 is a lower topological entropy variant of that task. Results indicated that given a high topological entropy grammar system, children with developmental dyslexia who were similar to the reading age-matched control group had substantial difficulty in performing the task as compared to typically developing children, who exhibited intact implicit learning of the grammar. On the other hand, when tested on a lower topological entropy grammar system, all groups performed above chance level, indicating that children with developmental dyslexia were able to identify rules from a given grammar system. The results reinforced the significance of graph complexity when experimenting with artificial grammar learning tasks, particularly with dyslexic participants.

Intelligent transportation systems for work zones : deployment benefits and lessons learned

DOT National Transportation Integrated Search

2000-12-01

This paper presents what has been learned in four principal areas of arterial management: 1) adaptive control strategies; 2) advanced traveler information systems; 3) automated enforcement; and 4) integration. The levels of deployment, benefits, depl...

Using Reinforcement Learning to Provide Stable Brain-Machine Interface Control Despite Neural Input Reorganization

PubMed Central

Pohlmeyer, Eric A.; Mahmoudi, Babak; Geng, Shijia; Prins, Noeline W.; Sanchez, Justin C.

2014-01-01

Brain-machine interface (BMI) systems give users direct neural control of robotic, communication, or functional electrical stimulation systems. As BMI systems begin transitioning from laboratory settings into activities of daily living, an important goal is to develop neural decoding algorithms that can be calibrated with a minimal burden on the user, provide stable control for long periods of time, and can be responsive to fluctuations in the decoder’s neural input space (e.g. neurons appearing or being lost amongst electrode recordings). These are significant challenges for static neural decoding algorithms that assume stationary input/output relationships. Here we use an actor-critic reinforcement learning architecture to provide an adaptive BMI controller that can successfully adapt to dramatic neural reorganizations, can maintain its performance over long time periods, and which does not require the user to produce specific kinetic or kinematic activities to calibrate the BMI. Two marmoset monkeys used the Reinforcement Learning BMI (RLBMI) to successfully control a robotic arm during a two-target reaching task. The RLBMI was initialized using random initial conditions, and it quickly learned to control the robot from brain states using only a binary evaluative feedback regarding whether previously chosen robot actions were good or bad. The RLBMI was able to maintain control over the system throughout sessions spanning multiple weeks. Furthermore, the RLBMI was able to quickly adapt and maintain control of the robot despite dramatic perturbations to the neural inputs, including a series of tests in which the neuron input space was deliberately halved or doubled. PMID:24498055

Impairments in learning by monetary rewards and alcohol-associated rewards in detoxified alcoholic patients.

PubMed

Jokisch, Daniel; Roser, Patrik; Juckel, Georg; Daum, Irene; Bellebaum, Christian

2014-07-01

Excessive alcohol consumption has been linked to structural and functional brain changes associated with cognitive, emotional, and behavioral impairments. It has been suggested that neural processing in the reward system is also affected by alcoholism. The present study aimed at further investigating reward-based associative learning and reversal learning in detoxified alcohol-dependent patients. Twenty-one detoxified alcohol-dependent patients and 26 healthy control subjects participated in a probabilistic learning task using monetary and alcohol-associated rewards as feedback stimuli indicating correct responses. Performance during acquisition and reversal learning in the different feedback conditions was analyzed. Alcohol-dependent patients and healthy control subjects showed an increase in learning performance over learning blocks during acquisition, with learning performance being significantly lower in alcohol-dependent patients. After changing the contingencies, alcohol-dependent patients exhibited impaired reversal learning and showed, in contrast to healthy controls, different learning curves for different types of rewards with no increase in performance for high monetary and alcohol-associated feedback. The present findings provide evidence that dysfunctional processing in the reward system in alcohol-dependent patients leads to alterations in reward-based learning resulting in a generally reduced performance. In addition, the results suggest that alcohol-dependent patients are, in particular, more impaired in changing an established behavior originally reinforced by high rewards. Copyright © 2014 by the Research Society on Alcoholism.

Learning feedback and feedforward control in a mirror-reversed visual environment.

PubMed

Kasuga, Shoko; Telgen, Sebastian; Ushiba, Junichi; Nozaki, Daichi; Diedrichsen, Jörn

2015-10-01

When we learn a novel task, the motor system needs to acquire both feedforward and feedback control. Currently, little is known about how the learning of these two mechanisms relate to each other. In the present study, we tested whether feedforward and feedback control need to be learned separately, or whether they are learned as common mechanism when a new control policy is acquired. Participants were trained to reach to two lateral and one central target in an environment with mirror (left-right)-reversed visual feedback. One group was allowed to make online movement corrections, whereas the other group only received visual information after the end of the movement. Learning of feedforward control was assessed by measuring the accuracy of the initial movement direction to lateral targets. Feedback control was measured in the responses to sudden visual perturbations of the cursor when reaching to the central target. Although feedforward control improved in both groups, it was significantly better when online corrections were not allowed. In contrast, feedback control only adaptively changed in participants who received online feedback and remained unchanged in the group without online corrections. Our findings suggest that when a new control policy is acquired, feedforward and feedback control are learned separately, and that there may be a trade-off in learning between feedback and feedforward controllers. Copyright © 2015 the American Physiological Society.

Learning feedback and feedforward control in a mirror-reversed visual environment

PubMed Central

Kasuga, Shoko; Telgen, Sebastian; Ushiba, Junichi; Nozaki, Daichi

2015-01-01

When we learn a novel task, the motor system needs to acquire both feedforward and feedback control. Currently, little is known about how the learning of these two mechanisms relate to each other. In the present study, we tested whether feedforward and feedback control need to be learned separately, or whether they are learned as common mechanism when a new control policy is acquired. Participants were trained to reach to two lateral and one central target in an environment with mirror (left-right)-reversed visual feedback. One group was allowed to make online movement corrections, whereas the other group only received visual information after the end of the movement. Learning of feedforward control was assessed by measuring the accuracy of the initial movement direction to lateral targets. Feedback control was measured in the responses to sudden visual perturbations of the cursor when reaching to the central target. Although feedforward control improved in both groups, it was significantly better when online corrections were not allowed. In contrast, feedback control only adaptively changed in participants who received online feedback and remained unchanged in the group without online corrections. Our findings suggest that when a new control policy is acquired, feedforward and feedback control are learned separately, and that there may be a trade-off in learning between feedback and feedforward controllers. PMID:26245313

From brain synapses to systems for learning and memory: Object recognition, spatial navigation, timed conditioning, and movement control.

PubMed

Grossberg, Stephen

2015-09-24

This article provides an overview of neural models of synaptic learning and memory whose expression in adaptive behavior depends critically on the circuits and systems in which the synapses are embedded. It reviews Adaptive Resonance Theory, or ART, models that use excitatory matching and match-based learning to achieve fast category learning and whose learned memories are dynamically stabilized by top-down expectations, attentional focusing, and memory search. ART clarifies mechanistic relationships between consciousness, learning, expectation, attention, resonance, and synchrony. ART models are embedded in ARTSCAN architectures that unify processes of invariant object category learning, recognition, spatial and object attention, predictive remapping, and eye movement search, and that clarify how conscious object vision and recognition may fail during perceptual crowding and parietal neglect. The generality of learned categories depends upon a vigilance process that is regulated by acetylcholine via the nucleus basalis. Vigilance can get stuck at too high or too low values, thereby causing learning problems in autism and medial temporal amnesia. Similar synaptic learning laws support qualitatively different behaviors: Invariant object category learning in the inferotemporal cortex; learning of grid cells and place cells in the entorhinal and hippocampal cortices during spatial navigation; and learning of time cells in the entorhinal-hippocampal system during adaptively timed conditioning, including trace conditioning. Spatial and temporal processes through the medial and lateral entorhinal-hippocampal system seem to be carried out with homologous circuit designs. Variations of a shared laminar neocortical circuit design have modeled 3D vision, speech perception, and cognitive working memory and learning. A complementary kind of inhibitory matching and mismatch learning controls movement. This article is part of a Special Issue entitled SI: Brain and Memory. Copyright © 2014 Elsevier B.V. All rights reserved.

Automatic learning rate adjustment for self-supervising autonomous robot control

NASA Technical Reports Server (NTRS)

Arras, Michael K.; Protzel, Peter W.; Palumbo, Daniel L.

1992-01-01

Described is an application in which an Artificial Neural Network (ANN) controls the positioning of a robot arm with five degrees of freedom by using visual feedback provided by two cameras. This application and the specific ANN model, local liner maps, are based on the work of Ritter, Martinetz, and Schulten. We extended their approach by generating a filtered, average positioning error from the continuous camera feedback and by coupling the learning rate to this error. When the network learns to position the arm, the positioning error decreases and so does the learning rate until the system stabilizes at a minimum error and learning rate. This abolishes the need for a predetermined cooling schedule. The automatic cooling procedure results in a closed loop control with no distinction between a learning phase and a production phase. If the positioning error suddenly starts to increase due to an internal failure such as a broken joint, or an environmental change such as a camera moving, the learning rate increases accordingly. Thus, learning is automatically activated and the network adapts to the new condition after which the error decreases again and learning is 'shut off'. The automatic cooling is therefore a prerequisite for the autonomy and the fault tolerance of the system.

Identification and real-time position control of a servo-hydraulic rotary actuator by means of a neurobiologically motivated algorithm.

PubMed

Sadeghieh, Ali; Sazgar, Hadi; Goodarzi, Kamyar; Lucas, Caro

2012-01-01

This paper presents a new intelligent approach for adaptive control of a nonlinear dynamic system. A modified version of the brain emotional learning based intelligent controller (BELBIC), a bio-inspired algorithm based upon a computational model of emotional learning which occurs in the amygdala, is utilized for position controlling a real laboratorial rotary electro-hydraulic servo (EHS) system. EHS systems are known to be nonlinear and non-smooth due to many factors such as leakage, friction, hysteresis, null shift, saturation, dead zone, and especially fluid flow expression through the servo valve. The large value of these factors can easily influence the control performance in the presence of a poor design. In this paper, a mathematical model of the EHS system is derived, and then the parameters of the model are identified using the recursive least squares method. In the next step, a BELBIC is designed based on this dynamic model and utilized to control the real laboratorial EHS system. To prove the effectiveness of the modified BELBIC's online learning ability in reducing the overall tracking error, results have been compared to those obtained from an optimal PID controller, an auto-tuned fuzzy PI controller (ATFPIC), and a neural network predictive controller (NNPC) under similar circumstances. The results demonstrate not only excellent improvement in control action, but also less energy consumption. Copyright © 2011 ISA. Published by Elsevier Ltd. All rights reserved.

Visually guided gait modifications for stepping over an obstacle: a bio-inspired approach.

PubMed

Silva, Pedro; Matos, Vitor; Santos, Cristina P

2014-02-01

There is an increasing interest in conceiving robotic systems that are able to move and act in an unstructured and not predefined environment, for which autonomy and adaptability are crucial features. In nature, animals are autonomous biological systems, which often serve as bio-inspiration models, not only for their physical and mechanical properties, but also their control structures that enable adaptability and autonomy-for which learning is (at least) partially responsible. This work proposes a system which seeks to enable a quadruped robot to online learn to detect and to avoid stumbling on an obstacle in its path. The detection relies in a forward internal model that estimates the robot's perceptive information by exploring the locomotion repetitive nature. The system adapts the locomotion in order to place the robot optimally before attempting to step over the obstacle, avoiding any stumbling. Locomotion adaptation is achieved by changing control parameters of a central pattern generator (CPG)-based locomotion controller. The mechanism learns the necessary alterations to the stride length in order to adapt the locomotion by changing the required CPG parameter. Both learning tasks occur online and together define a sensorimotor map, which enables the robot to learn to step over the obstacle in its path. Simulation results show the feasibility of the proposed approach.

Toward a dual-learning systems model of speech category learning

PubMed Central

Chandrasekaran, Bharath; Koslov, Seth R.; Maddox, W. T.

2014-01-01

More than two decades of work in vision posits the existence of dual-learning systems of category learning. The reflective system uses working memory to develop and test rules for classifying in an explicit fashion, while the reflexive system operates by implicitly associating perception with actions that lead to reinforcement. Dual-learning systems models hypothesize that in learning natural categories, learners initially use the reflective system and, with practice, transfer control to the reflexive system. The role of reflective and reflexive systems in auditory category learning and more specifically in speech category learning has not been systematically examined. In this article, we describe a neurobiologically constrained dual-learning systems theoretical framework that is currently being developed in speech category learning and review recent applications of this framework. Using behavioral and computational modeling approaches, we provide evidence that speech category learning is predominantly mediated by the reflexive learning system. In one application, we explore the effects of normal aging on non-speech and speech category learning. Prominently, we find a large age-related deficit in speech learning. The computational modeling suggests that older adults are less likely to transition from simple, reflective, unidimensional rules to more complex, reflexive, multi-dimensional rules. In a second application, we summarize a recent study examining auditory category learning in individuals with elevated depressive symptoms. We find a deficit in reflective-optimal and an enhancement in reflexive-optimal auditory category learning. Interestingly, individuals with elevated depressive symptoms also show an advantage in learning speech categories. We end with a brief summary and description of a number of future directions. PMID:25132827

Inductive Learning Approaches for Improving Pilot Awareness of Aircraft Faults

NASA Technical Reports Server (NTRS)

Spikovska, Lilly; Iverson, David L.; Poll, Scott; Pryor, anna

2005-01-01

Neural network flight controllers are able to accommodate a variety of aircraft control surface faults without detectable degradation of aircraft handling qualities. Under some faults, however, the effective flight envelope is reduced; this can lead to unexpected behavior if a pilot performs an action that exceeds the remaining control authority of the damaged aircraft. The goal of our work is to increase the pilot s situational awareness by informing him of the type of damage and resulting reduction in flight envelope. Our methodology integrates two inductive learning systems with novel visualization techniques. One learning system, the Inductive Monitoring System (IMS), learns to detect when a simulation includes faulty controls, while two others, Inductive Classification System (INCLASS) and multiple binary decision tree system (utilizing C4.5), determine the type of fault. In off-line training using only non-failure data, IMS constructs a characterization of nominal flight control performance based on control signals issued by the neural net flight controller. This characterization can be used to determine the degree of control augmentation required in the pitch, roll, and yaw command channels to counteract control surface failures. This derived information is typically sufficient to distinguish between the various control surface failures and is used to train both INCLASS and C4.5. Using data from failed control surface flight simulations, INCLASS and C4.5 independently discover and amplify features in IMS results that can be used to differentiate each distinct control surface failure situation. In real-time flight simulations, distinguishing features learned during training are used to classify control surface failures. Knowledge about the type of failure can be used by an additional automated system to alter its approach for planning tactical and strategic maneuvers. The knowledge can also be used directly to increase the pilot s situational awareness and inform manual maneuver decisions. Our multi-modal display of this information provides speech output to issue control surface failure warnings to a lesser-used communication channel and provides graphical displays with pilot-selectable !eve!s of details to issues additional information about the failure. We also describe a potential presentation for flight envelope reduction that can be viewed separately or integrated with an existing attitude indicator instrument. Preliminary results suggest that the inductive approach is capable of detecting that a control surface has failed and determining the type of fault. Furthermore, preliminary evaluations suggest that the interface discloses a concise summary of this information to the pilot.

Motor skill learning and offline-changes in TGA patients with acute hippocampal CA1 lesions.

PubMed

Döhring, Juliane; Stoldt, Anne; Witt, Karsten; Schönfeld, Robby; Deuschl, Günther; Born, Jan; Bartsch, Thorsten

2017-04-01

Learning and the formation of memory are reflected in various memory systems in the human brain such as the hippocampus based declarative memory system and the striatum-cortex based system involved in motor sequence learning. It is a matter of debate how both memory systems interact in humans during learning and consolidation and how this interaction is influenced by sleep. We studied the effect of an acute dysfunction of hippocampal CA1 neurons on the acquisition (on-line condition) and off-line changes of a motor skill in patients with a transient global amnesia (TGA). Sixteen patients (68 ± 4.4 yrs) were studied in the acute phase and during follow-up using a declarative and procedural test, and were compared to controls. Acute TGA patients displayed profound deficits in all declarative memory functions. During the acute amnestic phase, patients were able to acquire the motor skill task reflected by increasing finger tapping speed across the on-line condition, albeit to a lesser degree than during follow-up or compared to controls. Retrieval two days later indicated a greater off-line gain in motor speed in patients than controls. Moreover, this gain in motor skill performance was negatively correlated to the declarative learning deficit. Our results suggest a differential interaction between procedural and declarative memory systems during acquisition and consolidation of motor sequences in older humans. During acquisition, hippocampal dysfunction attenuates fast learning and thus unmasks the slow and rigid learning curve of striatum-based procedural learning. The stronger gains in the post-consolidation condition in motor skill in CA1 lesioned patients indicate a facilitated consolidation process probably occurring during sleep, and suggest a competitive interaction between the memory systems. These findings might be a reflection of network reorganization and plasticity in older humans and in the presence of CA1 hippocampal pathology. Copyright © 2016 Elsevier Ltd. All rights reserved.

Reinforcement-Learning-Based Robust Controller Design for Continuous-Time Uncertain Nonlinear Systems Subject to Input Constraints.

PubMed

Liu, Derong; Yang, Xiong; Wang, Ding; Wei, Qinglai

2015-07-01

The design of stabilizing controller for uncertain nonlinear systems with control constraints is a challenging problem. The constrained-input coupled with the inability to identify accurately the uncertainties motivates the design of stabilizing controller based on reinforcement-learning (RL) methods. In this paper, a novel RL-based robust adaptive control algorithm is developed for a class of continuous-time uncertain nonlinear systems subject to input constraints. The robust control problem is converted to the constrained optimal control problem with appropriately selecting value functions for the nominal system. Distinct from typical action-critic dual networks employed in RL, only one critic neural network (NN) is constructed to derive the approximate optimal control. Meanwhile, unlike initial stabilizing control often indispensable in RL, there is no special requirement imposed on the initial control. By utilizing Lyapunov's direct method, the closed-loop optimal control system and the estimated weights of the critic NN are proved to be uniformly ultimately bounded. In addition, the derived approximate optimal control is verified to guarantee the uncertain nonlinear system to be stable in the sense of uniform ultimate boundedness. Two simulation examples are provided to illustrate the effectiveness and applicability of the present approach.

Design of Sensors for Control of Closed Loop Life Support Systems

NASA Technical Reports Server (NTRS)

1990-01-01

A brief summary is presented of a Engineering Design sequence, a cooperation between NASA-Kennedy and the University of Florida on the Controlled Environmental Life Support System (CELSS) program. Part of the class was devoted to learning general principles and techniques of design. The next portion of the class was devoted to learning to design, actually fabricating and testing small components and subsystems of a CELSS.

A Study of a Social Annotation Modeling Learning System

ERIC Educational Resources Information Center

Samuel, Roy David; Kim, Chanmin; Johnson, Tristan E.

2011-01-01

The transition from classroom instruction to e-learning raises pedagogical challenges for university instructors. A controlled integration of e-learning tools into classroom instruction may offer learners tangible benefits and improved effectiveness. This design-based research (DBR) study engaged students in e-learning activities integrated into…

Approximately adaptive neural cooperative control for nonlinear multiagent systems with performance guarantee

NASA Astrophysics Data System (ADS)

Wang, Jing; Yang, Tianyu; Staskevich, Gennady; Abbe, Brian

2017-04-01

This paper studies the cooperative control problem for a class of multiagent dynamical systems with partially unknown nonlinear system dynamics. In particular, the control objective is to solve the state consensus problem for multiagent systems based on the minimisation of certain cost functions for individual agents. Under the assumption that there exist admissible cooperative controls for such class of multiagent systems, the formulated problem is solved through finding the optimal cooperative control using the approximate dynamic programming and reinforcement learning approach. With the aid of neural network parameterisation and online adaptive learning, our method renders a practically implementable approximately adaptive neural cooperative control for multiagent systems. Specifically, based on the Bellman's principle of optimality, the Hamilton-Jacobi-Bellman (HJB) equation for multiagent systems is first derived. We then propose an approximately adaptive policy iteration algorithm for multiagent cooperative control based on neural network approximation of the value functions. The convergence of the proposed algorithm is rigorously proved using the contraction mapping method. The simulation results are included to validate the effectiveness of the proposed algorithm.

Cardiovascular Physiology for First-Year Medical Students: Teaching and Learning through Games.

ERIC Educational Resources Information Center

France, Vanetia M.

1978-01-01

Describes a card game designed to help medical students learn to manipulate concepts fundamental to the functions of the cardiovascular system (CVS) and to understand the interrelationships between different controlled variables in the system. (Author/MA)

The Impact of Pictorial Display on Operator Learning and Performance. M.S. Thesis

NASA Technical Reports Server (NTRS)

Miller, R. A.; Messing, L. J.; Jagacinski, R. J.

1984-01-01

The effects of pictorially displayed information on human learning and performance of a simple control task were investigated. The controlled system was a harmonic oscillator and the system response was displayed to subjects as either an animated pendulum or a horizontally moving dot. Results indicated that the pendulum display did not effect performance scores but did significantly effect the learning processes of individual operators. The subjects with the pendulum display demonstrated more vertical internal models early in the experiment and the manner in which their internal models were tuned with practice showed increased variability between subjects.

Open Learning Systems in Further Education. Some Aspects of Resource Management. Coombe Lodge Working Paper. Information Bank Number 1611.

ERIC Educational Resources Information Center

Birch, D. W.; Latcham, J.

Open learning systems require the development of learning materials. Potential economies of scale and quality control advantages argue in favor of the central development of materials packages. Instructors would be free to concentrate upon the provision of tutorial and counseling support and laboratory and workshop hands-on experience. Some method…

Development of Constellation's Launch Control System

NASA Technical Reports Server (NTRS)

Lougheed, Kirk D.; Peaden, Cary J.

2010-01-01

The paper focuses on the National Aeronautics and Space Administration (NASA) Constellation Program's Launch Control System (LCS) development effort at Kennedy Space Center (KSC). It provides a brief history of some preceding efforts to provide launch control and ground processing systems for other NASA programs, and some lessons learned from those experiences. It then provides high level descriptions of the LCS mission, objectives, organization, architecture, and progress. It discusses some of our development tenets, including our use of standards based design and use of off-the-shelf products whenever possible, incremental development cycles, and highly reliable, available, and supportable enterprise class system servers. It concludes with some new lessons learned and our plans for the future.

Intelligent control and cooperation for mobile robots

NASA Astrophysics Data System (ADS)

Stingu, Petru Emanuel

The topic discussed in this work addresses the current research being conducted at the Automation & Robotics Research Institute in the areas of UAV quadrotor control and heterogenous multi-vehicle cooperation. Autonomy can be successfully achieved by a robot under the following conditions: the robot has to be able to acquire knowledge about the environment and itself, and it also has to be able to reason under uncertainty. The control system must react quickly to immediate challenges, but also has to slowly adapt and improve based on accumulated knowledge. The major contribution of this work is the transfer of the ADP algorithms from the purely theoretical environment to the complex real-world robotic platforms that work in real-time and in uncontrolled environments. Many solutions are adopted from those present in nature because they have been proven to be close to optimal in very different settings. For the control of a single platform, reinforcement learning algorithms are used to design suboptimal controllers for a class of complex systems that can be conceptually split in local loops with simpler dynamics and relatively weak coupling to the rest of the system. Optimality is enforced by having a global critic but the curse of dimensionality is avoided by using local actors and intelligent pre-processing of the information used for learning the optimal controllers. The system model is used for constructing the structure of the control system, but on top of that the adaptive neural networks that form the actors use the knowledge acquired during normal operation to get closer to optimal control. In real-world experiments, efficient learning is a strong requirement for success. This is accomplished by using an approximation of the system model to focus the learning for equivalent configurations of the state space. Due to the availability of only local data for training, neural networks with local activation functions are implemented. For the control of a formation of robots subjected to dynamic communication constraints, game theory is used in addition to reinforcement learning. The nodes maintain an extra set of state variables about all the other nodes that they can communicate to. The more important are trust and predictability. They are a way to incorporate knowledge acquired in the past into the control decisions taken by each node. The trust variable provides a simple mechanism for the implementation of reinforcement learning. For robot formations, potential field based control algorithms are used to generate the control commands. The formation structure changes due to the environment and due to the decisions of the nodes. It is a problem of building a graph and coalitions by having distributed decisions but still reaching an optimal behavior globally.

Selected Flight Test Results for Online Learning Neural Network-Based Flight Control System

NASA Technical Reports Server (NTRS)

Williams-Hayes, Peggy S.

2004-01-01

The NASA F-15 Intelligent Flight Control System project team developed a series of flight control concepts designed to demonstrate neural network-based adaptive controller benefits, with the objective to develop and flight-test control systems using neural network technology to optimize aircraft performance under nominal conditions and stabilize the aircraft under failure conditions. This report presents flight-test results for an adaptive controller using stability and control derivative values from an online learning neural network. A dynamic cell structure neural network is used in conjunction with a real-time parameter identification algorithm to estimate aerodynamic stability and control derivative increments to baseline aerodynamic derivatives in flight. This open-loop flight test set was performed in preparation for a future phase in which the learning neural network and parameter identification algorithm output would provide the flight controller with aerodynamic stability and control derivative updates in near real time. Two flight maneuvers are analyzed - pitch frequency sweep and automated flight-test maneuver designed to optimally excite the parameter identification algorithm in all axes. Frequency responses generated from flight data are compared to those obtained from nonlinear simulation runs. Flight data examination shows that addition of flight-identified aerodynamic derivative increments into the simulation improved aircraft pitch handling qualities.

Agent-based traffic management and reinforcement learning in congested intersection network.

DOT National Transportation Integrated Search

2012-08-01

This study evaluates the performance of traffic control systems based on reinforcement learning (RL), also called approximate dynamic programming (ADP). Two algorithms have been selected for testing: 1) Q-learning and 2) approximate dynamic programmi...

Learning and Understanding System Stability Using Illustrative Dynamic Texture Examples

ERIC Educational Resources Information Center

Liu, Huaping; Xiao, Wei; Zhao, Hongyan; Sun, Fuchun

2014-01-01

System stability is a basic concept in courses on dynamic system analysis and control for undergraduate students with computer science backgrounds. Typically, this was taught using a simple simulation example of an inverted pendulum. Unfortunately, many difficult issues arise in the learning and understanding of the concepts of stability,…

Comparison of Computer-Based Versus Counselor-Based Occupational Information Systems with Disadvantaged Vocational Students

ERIC Educational Resources Information Center

Maola, Joseph; Kane, Gary

1976-01-01

Subjects, who were Occupational Work Experience students, were randomly assigned to individual guidance from either a computerized occupational information system, to a counselor-based information system or to a control group. Results demonstrate a hierarchical learning effect: The computer group learned more than the counseled group, which…

Mountain Plains Learning Experience Guide: Automotive Repair. Course: Emission Systems.

ERIC Educational Resources Information Center

Schramm, C.; Osland, Walt

One of twelve individualized courses included in an automotive repair curriculum, this course covers the theory, testing, and servicing of automotive emission control systems. The course is comprised of one unit, Fundamentals of Emission Systems. The unit begins with a Unit Learning Experience Guide that gives directions for unit completion. The…

Agents Control in Intelligent Learning Systems: The Case of Reactive Characteristics

ERIC Educational Resources Information Center

Laureano-Cruces, Ana Lilia; Ramirez-Rodriguez, Javier; de Arriaga, Fernando; Escarela-Perez, Rafael

2006-01-01

Intelligent learning systems (ILSs) have evolved in the last few years basically because of influences received from multi-agent architectures (MAs). Conflict resolution among agents has been a very important problem for multi-agent systems, with specific features in the case of ILSs. The literature shows that ILSs with cognitive or pedagogical…

Open-closed-loop iterative learning control for a class of nonlinear systems with random data dropouts

NASA Astrophysics Data System (ADS)

Cheng, X. Y.; Wang, H. B.; Jia, Y. L.; Dong, YH

2018-05-01

In this paper, an open-closed-loop iterative learning control (ILC) algorithm is constructed for a class of nonlinear systems subjecting to random data dropouts. The ILC algorithm is implemented by a networked control system (NCS), where only the off-line data is transmitted by network while the real-time data is delivered in the point-to-point way. Thus, there are two controllers rather than one in the control system, which makes better use of the saved and current information and thereby improves the performance achieved by open-loop control alone. During the transfer of off-line data between the nonlinear plant and the remote controller data dropout occurs randomly and the data dropout rate is modeled as a binary Bernoulli random variable. Both measurement and control data dropouts are taken into consideration simultaneously. The convergence criterion is derived based on rigorous analysis. Finally, the simulation results verify the effectiveness of the proposed method.

Adaptive Fuzzy Bounded Control for Consensus of Multiple Strict-Feedback Nonlinear Systems.

PubMed

Wang, Wei; Tong, Shaocheng

2018-02-01

This paper studies the adaptive fuzzy bounded control problem for leader-follower multiagent systems, where each follower is modeled by the uncertain nonlinear strict-feedback system. Combining the fuzzy approximation with the dynamic surface control, an adaptive fuzzy control scheme is developed to guarantee the output consensus of all agents under directed communication topologies. Different from the existing results, the bounds of the control inputs are known as a priori, and they can be determined by the feedback control gains. To realize smooth and fast learning, a predictor is introduced to estimate each error surface, and the corresponding predictor error is employed to learn the optimal fuzzy parameter vector. It is proved that the developed adaptive fuzzy control scheme guarantees the uniformly ultimate boundedness of the closed-loop systems, and the tracking error converges to a small neighborhood of the origin. The simulation results and comparisons are provided to show the validity of the control strategy presented in this paper.

Musical Sequence Learning and EEG Correlates of Audiomotor Processing

PubMed Central

Schalles, Matt D.; Pineda, Jaime A.

2015-01-01

Our motor and auditory systems are functionally connected during musical performance, and functional imaging suggests that the association is strong enough that passive music listening can engage the motor system. As predictive coding constrains movement sequence selections, could the motor system contribute to sequential processing of musical passages? If this is the case, then we hypothesized that the motor system should respond preferentially to passages of music that contain similar sequential information, even if other aspects of music, such as the absolute pitch, have been altered. We trained piano naive subjects with a learn-to play-by-ear paradigm, to play a simple melodic sequence over five days. After training, we recorded EEG of subjects listening to the song they learned to play, a transposed version of that song, and a control song with different notes and sequence from the learned song. Beta band power over sensorimotor scalp showed increased suppression for the learned song, a moderate level of suppression for the transposed song, and no suppression for the control song. As beta power is associated with attention and motor processing, we interpret this as support of the motor system's activity during covert perception of music one can play and similar musical sequences. PMID:26527118

Building Knowledge Structures by Testing Helps Children With Mathematical Learning Difficulty.

PubMed

Zhang, Yiyun; Zhou, Xinlin

2016-01-01

Mathematical learning difficulty (MLD) is prevalent in the development of mathematical abilities. Previous interventions for children with MLD have focused on number sense or basic mathematical skills. This study investigated whether mathematical performance of fifth grade children with MLD could be improved by developing knowledge structures by testing using a web-based curriculum learning system. A total of 142 children with MLD were recruited; half of the children were in the experimental group (using the system), and the other half were in the control group (not using the system). The children were encouraged to use the web-based learning system at home for at least a 15-min session, at least once a week, for one and a half months. The mean accumulated time of testing on the system for children in the experimental group was 56.2 min. Children in the experimental group had significantly higher scores on their final mathematical examination compared to the control group. The results suggest that web-based curriculum learning through testing that promotes the building of knowledge structures for a mathematical course was helpful for children with MLD. © Hammill Institute on Disabilities 2014.

A Control Systems Concept Inventory Test Design and Assessment

ERIC Educational Resources Information Center

Bristow, M.; Erkorkmaz, K.; Huissoon, J. P.; Jeon, Soo; Owen, W. S.; Waslander, S. L.; Stubley, G. D.

2012-01-01

Any meaningful initiative to improve the teaching and learning in introductory control systems courses needs a clear test of student conceptual understanding to determine the effectiveness of proposed methods and activities. The authors propose a control systems concept inventory. Development of the inventory was collaborative and iterative. The…

Interactions Among Working Memory, Reinforcement Learning, and Effort in Value-Based Choice: A New Paradigm and Selective Deficits in Schizophrenia.

PubMed

Collins, Anne G E; Albrecht, Matthew A; Waltz, James A; Gold, James M; Frank, Michael J

2017-09-15

When studying learning, researchers directly observe only the participants' choices, which are often assumed to arise from a unitary learning process. However, a number of separable systems, such as working memory (WM) and reinforcement learning (RL), contribute simultaneously to human learning. Identifying each system's contributions is essential for mapping the neural substrates contributing in parallel to behavior; computational modeling can help to design tasks that allow such a separable identification of processes and infer their contributions in individuals. We present a new experimental protocol that separately identifies the contributions of RL and WM to learning, is sensitive to parametric variations in both, and allows us to investigate whether the processes interact. In experiments 1 and 2, we tested this protocol with healthy young adults (n = 29 and n = 52, respectively). In experiment 3, we used it to investigate learning deficits in medicated individuals with schizophrenia (n = 49 patients, n = 32 control subjects). Experiments 1 and 2 established WM and RL contributions to learning, as evidenced by parametric modulations of choice by load and delay and reward history, respectively. They also showed interactions between WM and RL, where RL was enhanced under high WM load. Moreover, we observed a cost of mental effort when controlling for reinforcement history: participants preferred stimuli they encountered under low WM load. Experiment 3 revealed selective deficits in WM contributions and preserved RL value learning in individuals with schizophrenia compared with control subjects. Computational approaches allow us to disentangle contributions of multiple systems to learning and, consequently, to further our understanding of psychiatric diseases. Copyright © 2017 Society of Biological Psychiatry. Published by Elsevier Inc. All rights reserved.

A novel model of motor learning capable of developing an optimal movement control law online from scratch.

PubMed

Shimansky, Yury P; Kang, Tao; He, Jiping

2004-02-01

A computational model of a learning system (LS) is described that acquires knowledge and skill necessary for optimal control of a multisegmental limb dynamics (controlled object or CO), starting from "knowing" only the dimensionality of the object's state space. It is based on an optimal control problem setup different from that of reinforcement learning. The LS solves the optimal control problem online while practicing the manipulation of CO. The system's functional architecture comprises several adaptive components, each of which incorporates a number of mapping functions approximated based on artificial neural nets. Besides the internal model of the CO's dynamics and adaptive controller that computes the control law, the LS includes a new type of internal model, the minimal cost (IM(mc)) of moving the controlled object between a pair of states. That internal model appears critical for the LS's capacity to develop an optimal movement trajectory. The IM(mc) interacts with the adaptive controller in a cooperative manner. The controller provides an initial approximation of an optimal control action, which is further optimized in real time based on the IM(mc). The IM(mc) in turn provides information for updating the controller. The LS's performance was tested on the task of center-out reaching to eight randomly selected targets with a 2DOF limb model. The LS reached an optimal level of performance in a few tens of trials. It also quickly adapted to movement perturbations produced by two different types of external force field. The results suggest that the proposed design of a self-optimized control system can serve as a basis for the modeling of motor learning that includes the formation and adaptive modification of the plan of a goal-directed movement.

Intelligent automated control of life support systems using proportional representations.

PubMed

Wu, Annie S; Garibay, Ivan I

2004-06-01

Effective automatic control of Advanced Life Support Systems (ALSS) is a crucial component of space exploration. An ALSS is a coupled dynamical system which can be extremely sensitive and difficult to predict. As a result, such systems can be difficult to control using deliberative and deterministic methods. We investigate the performance of two machine learning algorithms, a genetic algorithm (GA) and a stochastic hill-climber (SH), on the problem of learning how to control an ALSS, and compare the impact of two different types of problem representations on the performance of both algorithms. We perform experiments on three ALSS optimization problems using five strategies with multiple variations of a proportional representation for a total of 120 experiments. Results indicate that although a proportional representation can effectively boost GA performance, it does not necessarily have the same effect on other algorithms such as SH. Results also support previous conclusions that multivector control strategies are an effective method for control of coupled dynamical systems.

Generalization in Adaptation to Stable and Unstable Dynamics

PubMed Central

Kadiallah, Abdelhamid; Franklin, David W.; Burdet, Etienne

2012-01-01

Humans skillfully manipulate objects and tools despite the inherent instability. In order to succeed at these tasks, the sensorimotor control system must build an internal representation of both the force and mechanical impedance. As it is not practical to either learn or store motor commands for every possible future action, the sensorimotor control system generalizes a control strategy for a range of movements based on learning performed over a set of movements. Here, we introduce a computational model for this learning and generalization, which specifies how to learn feedforward muscle activity in a function of the state space. Specifically, by incorporating co-activation as a function of error into the feedback command, we are able to derive an algorithm from a gradient descent minimization of motion error and effort, subject to maintaining a stability margin. This algorithm can be used to learn to coordinate any of a variety of motor primitives such as force fields, muscle synergies, physical models or artificial neural networks. This model for human learning and generalization is able to adapt to both stable and unstable dynamics, and provides a controller for generating efficient adaptive motor behavior in robots. Simulation results exhibit predictions consistent with all experiments on learning of novel dynamics requiring adaptation of force and impedance, and enable us to re-examine some of the previous interpretations of experiments on generalization. PMID:23056191

Off-Policy Actor-Critic Structure for Optimal Control of Unknown Systems With Disturbances.

PubMed

Song, Ruizhuo; Lewis, Frank L; Wei, Qinglai; Zhang, Huaguang

2016-05-01

An optimal control method is developed for unknown continuous-time systems with unknown disturbances in this paper. The integral reinforcement learning (IRL) algorithm is presented to obtain the iterative control. Off-policy learning is used to allow the dynamics to be completely unknown. Neural networks are used to construct critic and action networks. It is shown that if there are unknown disturbances, off-policy IRL may not converge or may be biased. For reducing the influence of unknown disturbances, a disturbances compensation controller is added. It is proven that the weight errors are uniformly ultimately bounded based on Lyapunov techniques. Convergence of the Hamiltonian function is also proven. The simulation study demonstrates the effectiveness of the proposed optimal control method for unknown systems with disturbances.

Tunnel Ventilation Control Using Reinforcement Learning Methodology

NASA Astrophysics Data System (ADS)

Chu, Baeksuk; Kim, Dongnam; Hong, Daehie; Park, Jooyoung; Chung, Jin Taek; Kim, Tae-Hyung

The main purpose of tunnel ventilation system is to maintain CO pollutant concentration and VI (visibility index) under an adequate level to provide drivers with comfortable and safe driving environment. Moreover, it is necessary to minimize power consumption used to operate ventilation system. To achieve the objectives, the control algorithm used in this research is reinforcement learning (RL) method. RL is a goal-directed learning of a mapping from situations to actions without relying on exemplary supervision or complete models of the environment. The goal of RL is to maximize a reward which is an evaluative feedback from the environment. In the process of constructing the reward of the tunnel ventilation system, two objectives listed above are included, that is, maintaining an adequate level of pollutants and minimizing power consumption. RL algorithm based on actor-critic architecture and gradient-following algorithm is adopted to the tunnel ventilation system. The simulations results performed with real data collected from existing tunnel ventilation system and real experimental verification are provided in this paper. It is confirmed that with the suggested controller, the pollutant level inside the tunnel was well maintained under allowable limit and the performance of energy consumption was improved compared to conventional control scheme.

The Impact of Team-Based Learning on Nervous System Examination Knowledge of Nursing Students.

PubMed

Hemmati Maslakpak, Masomeh; Parizad, Naser; Zareie, Farzad

2015-12-01

Team-based learning is one of the active learning approaches in which independent learning is combined with small group discussion in the class. This study aimed to determine the impact of team-based learning in nervous system examination knowledge of nursing students. This quasi-experimental study was conducted on 3(rd) grade nursing students, including 5th semester (intervention group) and 6(th) semester (control group). The traditional lecture method and the team-based learning method were used for educating the examination of the nervous system for intervention and control groups, respectively. The data were collected by a test covering 40-questions (multiple choice, matching, gap-filling and descriptive questions) before and after intervention in both groups. Individual Readiness Assurance Test (RAT) and Group Readiness Assurance Test (GRAT) used to collect data in the intervention group. In the end, the collected data were analyzed by SPSS ver. 13 using descriptive and inferential statistical tests. In team-based learning group, mean and standard deviation was 13.39 (4.52) before the intervention, which had been increased to 31.07 (3.20) after the intervention and this increase was statistically significant. Also, there was a statistically significant difference between the scores of RAT and GRAT in team-based learning group. Using team-based learning approach resulted in much better improvement and stability in the nervous system examination knowledge of nursing students compared to traditional lecture method; therefore, this method could be efficiently used as an effective educational approach in nursing education.

Working memory contributions to reinforcement learning impairments in schizophrenia.

PubMed

Collins, Anne G E; Brown, Jaime K; Gold, James M; Waltz, James A; Frank, Michael J

2014-10-08

Previous research has shown that patients with schizophrenia are impaired in reinforcement learning tasks. However, behavioral learning curves in such tasks originate from the interaction of multiple neural processes, including the basal ganglia- and dopamine-dependent reinforcement learning (RL) system, but also prefrontal cortex-dependent cognitive strategies involving working memory (WM). Thus, it is unclear which specific system induces impairments in schizophrenia. We recently developed a task and computational model allowing us to separately assess the roles of RL (slow, cumulative learning) mechanisms versus WM (fast but capacity-limited) mechanisms in healthy adult human subjects. Here, we used this task to assess patients' specific sources of impairments in learning. In 15 separate blocks, subjects learned to pick one of three actions for stimuli. The number of stimuli to learn in each block varied from two to six, allowing us to separate influences of capacity-limited WM from the incremental RL system. As expected, both patients (n = 49) and healthy controls (n = 36) showed effects of set size and delay between stimulus repetitions, confirming the presence of working memory effects. Patients performed significantly worse than controls overall, but computational model fits and behavioral analyses indicate that these deficits could be entirely accounted for by changes in WM parameters (capacity and reliability), whereas RL processes were spared. These results suggest that the working memory system contributes strongly to learning impairments in schizophrenia. Copyright © 2014 the authors 0270-6474/14/3413747-10$15.00/0.

Representing and Learning Complex Object Interactions

PubMed Central

Zhou, Yilun; Konidaris, George

2017-01-01

We present a framework for representing scenarios with complex object interactions, in which a robot cannot directly interact with the object it wishes to control, but must instead do so via intermediate objects. For example, a robot learning to drive a car can only indirectly change its pose, by rotating the steering wheel. We formalize such complex interactions as chains of Markov decision processes and show how they can be learned and used for control. We describe two systems in which a robot uses learning from demonstration to achieve indirect control: playing a computer game, and using a hot water dispenser to heat a cup of water. PMID:28593181

Cogging effect minimization in PMSM position servo system using dual high-order periodic adaptive learning compensation.

PubMed

Luo, Ying; Chen, Yangquan; Pi, Youguo

2010-10-01

Cogging effect which can be treated as a type of position-dependent periodic disturbance, is a serious disadvantage of the permanent magnetic synchronous motor (PMSM). In this paper, based on a simulation system model of PMSM position servo control, the cogging force, viscous friction, and applied load in the real PMSM control system are considered and presented. A dual high-order periodic adaptive learning compensation (DHO-PALC) method is proposed to minimize the cogging effect on the PMSM position and velocity servo system. In this DHO-PALC scheme, more than one previous periods stored information of both the composite tracking error and the estimate of the cogging force is used for the control law updating. Asymptotical stability proof with the proposed DHO-PALC scheme is presented. Simulation is implemented on the PMSM servo system model to illustrate the proposed method. When the constant speed reference is applied, the DHO-PALC can achieve a faster learning convergence speed than the first-order periodic adaptive learning compensation (FO-PALC). Moreover, when the designed reference signal changes periodically, the proposed DHO-PALC can obtain not only faster convergence speed, but also much smaller final error bound than the FO-PALC. Copyright © 2010 ISA. Published by Elsevier Ltd. All rights reserved.

Adaptive Fuzzy Systems in Computational Intelligence

NASA Technical Reports Server (NTRS)

Berenji, Hamid R.

1996-01-01

In recent years, the interest in computational intelligence techniques, which currently includes neural networks, fuzzy systems, and evolutionary programming, has grown significantly and a number of their applications have been developed in the government and industry. In future, an essential element in these systems will be fuzzy systems that can learn from experience by using neural network in refining their performances. The GARIC architecture, introduced earlier, is an example of a fuzzy reinforcement learning system which has been applied in several control domains such as cart-pole balancing, simulation of to Space Shuttle orbital operations, and tether control. A number of examples from GARIC's applications in these domains will be demonstrated.

The Roles of Feedback and Feedforward as Humans Learn to Control Unknown Dynamic Systems.

PubMed

Zhang, Xingye; Wang, Shaoqian; Hoagg, Jesse B; Seigler, T Michael

2018-02-01

We present results from an experiment in which human subjects interact with an unknown dynamic system 40 times during a two-week period. During each interaction, subjects are asked to perform a command-following (i.e., pursuit tracking) task. Each subject's performance at that task improves from the first trial to the last trial. For each trial, we use subsystem identification to estimate each subject's feedforward (or anticipatory) control, feedback (or reactive) control, and feedback time delay. Over the 40 trials, the magnitudes of the identified feedback controllers and the identified feedback time delays do not change significantly. In contrast, the identified feedforward controllers do change significantly. By the last trial, the average identified feedforward controller approximates the inverse of the dynamic system. This observation provides evidence that a fundamental component of human learning is updating the anticipatory control until it models the inverse dynamics.

Collective learning for the emergence of social norms in networked multiagent systems.

PubMed

Yu, Chao; Zhang, Minjie; Ren, Fenghui

2014-12-01

Social norms such as social rules and conventions play a pivotal role in sustaining system order by regulating and controlling individual behaviors toward a global consensus in large-scale distributed systems. Systematic studies of efficient mechanisms that can facilitate the emergence of social norms enable us to build and design robust distributed systems, such as electronic institutions and norm-governed sensor networks. This paper studies the emergence of social norms via learning from repeated local interactions in networked multiagent systems. A collective learning framework, which imitates the opinion aggregation process in human decision making, is proposed to study the impact of agent local collective behaviors on the emergence of social norms in a number of different situations. In the framework, each agent interacts repeatedly with all of its neighbors. At each step, an agent first takes a best-response action toward each of its neighbors and then combines all of these actions into a final action using ensemble learning methods. Extensive experiments are carried out to evaluate the framework with respect to different network topologies, learning strategies, numbers of actions, influences of nonlearning agents, and so on. Experimental results reveal some significant insights into the manipulation and control of norm emergence in networked multiagent systems achieved through local collective behaviors.

MCMAC-cVT: a novel on-line associative memory based CVT transmission control system.

PubMed

Ang, K K; Quek, C; Wahab, A

2002-03-01

This paper describes a novel application of an associative memory called the Modified Cerebellar Articulation Controller (MCMAC) (Int. J. Artif. Intell. Engng, 10 (1996) 135) in a continuous variable transmission (CVT) control system. It allows the on-line tuning of the associative memory and produces an effective gain-schedule for the automatic selection of the CVT gear ratio. Various control algorithms are investigated to control the CVT gear ratio to maintain the engine speed within a narrow range of efficient operating speed independently of the vehicle velocity. Extensive simulation results are presented to evaluate the control performance of a direct digital PID control algorithm with auto-tuning (Trans. ASME, 64 (1942)) and anti-windup mechanism. In particular, these results are contrasted against the control performance produced using the MCMAC (Int. J. Artif. Intell. Engng, 10 (1996) 135) with momentum, neighborhood learning and Averaged Trapezoidal Output (MCMAC-ATO) as the neural control algorithm for controlling the CVT. Simulation results are presented that show the reduced control fluctuations and improved learning capability of the MCMAC-ATO without incurring greater memory requirement. In particular, MCMAC-ATO is able to learn and control the CVT simultaneously while still maintaining acceptable control performance.

Adaptive Constrained Optimal Control Design for Data-Based Nonlinear Discrete-Time Systems With Critic-Only Structure.

PubMed

Luo, Biao; Liu, Derong; Wu, Huai-Ning

2018-06-01

Reinforcement learning has proved to be a powerful tool to solve optimal control problems over the past few years. However, the data-based constrained optimal control problem of nonaffine nonlinear discrete-time systems has rarely been studied yet. To solve this problem, an adaptive optimal control approach is developed by using the value iteration-based Q-learning (VIQL) with the critic-only structure. Most of the existing constrained control methods require the use of a certain performance index and only suit for linear or affine nonlinear systems, which is unreasonable in practice. To overcome this problem, the system transformation is first introduced with the general performance index. Then, the constrained optimal control problem is converted to an unconstrained optimal control problem. By introducing the action-state value function, i.e., Q-function, the VIQL algorithm is proposed to learn the optimal Q-function of the data-based unconstrained optimal control problem. The convergence results of the VIQL algorithm are established with an easy-to-realize initial condition . To implement the VIQL algorithm, the critic-only structure is developed, where only one neural network is required to approximate the Q-function. The converged Q-function obtained from the critic-only VIQL method is employed to design the adaptive constrained optimal controller based on the gradient descent scheme. Finally, the effectiveness of the developed adaptive control method is tested on three examples with computer simulation.

The Effectiveness of the Game-Based Learning System for the Improvement of American Sign Language Using Kinect

ERIC Educational Resources Information Center

Kamnardsiri, Teerawat; Hongsit, Ler-on; Khuwuthyakorn, Pattaraporn; Wongta, Noppon

2017-01-01

This paper investigated students' achievement for learning American Sign Language (ASL), using two different methods. There were two groups of samples. The first experimental group (Group A) was the game-based learning for ASL, using Kinect. The second control learning group (Group B) was the traditional face-to-face learning method, generally…

Cost-Benefit Arbitration Between Multiple Reinforcement-Learning Systems.

PubMed

Kool, Wouter; Gershman, Samuel J; Cushman, Fiery A

2017-09-01

Human behavior is sometimes determined by habit and other times by goal-directed planning. Modern reinforcement-learning theories formalize this distinction as a competition between a computationally cheap but inaccurate model-free system that gives rise to habits and a computationally expensive but accurate model-based system that implements planning. It is unclear, however, how people choose to allocate control between these systems. Here, we propose that arbitration occurs by comparing each system's task-specific costs and benefits. To investigate this proposal, we conducted two experiments showing that people increase model-based control when it achieves greater accuracy than model-free control, and especially when the rewards of accurate performance are amplified. In contrast, they are insensitive to reward amplification when model-based and model-free control yield equivalent accuracy. This suggests that humans adaptively balance habitual and planned action through on-line cost-benefit analysis.

Magnetic induction of hyperthermia by a modified self-learning fuzzy temperature controller

NASA Astrophysics Data System (ADS)

Wang, Wei-Cheng; Tai, Cheng-Chi

2017-07-01

The aim of this study involved developing a temperature controller for magnetic induction hyperthermia (MIH). A closed-loop controller was applied to track a reference model to guarantee a desired temperature response. The MIH system generated an alternating magnetic field to heat a high magnetic permeability material. This wireless induction heating had few side effects when it was extensively applied to cancer treatment. The effects of hyperthermia strongly depend on the precise control of temperature. However, during the treatment process, the control performance is degraded due to severe perturbations and parameter variations. In this study, a modified self-learning fuzzy logic controller (SLFLC) with a gain tuning mechanism was implemented to obtain high control performance in a wide range of treatment situations. This implementation was performed by appropriately altering the output scaling factor of a fuzzy inverse model to adjust the control rules. In this study, the proposed SLFLC was compared to the classical self-tuning fuzzy logic controller and fuzzy model reference learning control. Additionally, the proposed SLFLC was verified by conducting in vitro experiments with porcine liver. The experimental results indicated that the proposed controller showed greater robustness and excellent adaptability with respect to the temperature control of the MIH system.

Learning-based controller for biotechnology processing, and method of using

DOEpatents

Johnson, John A.; Stoner, Daphne L.; Larsen, Eric D.; Miller, Karen S.; Tolle, Charles R.

2004-09-14

The present invention relates to process control where some of the controllable parameters are difficult or impossible to characterize. The present invention relates to process control in biotechnology of such systems, but not limited to. Additionally, the present invention relates to process control in biotechnology minerals processing. In the inventive method, an application of the present invention manipulates a minerals bioprocess to find local exterma (maxima or minima) for selected output variables/process goals by using a learning-based controller for bioprocess oxidation of minerals during hydrometallurgical processing. The learning-based controller operates with or without human supervision and works to find processor optima without previously defined optima due to the non-characterized nature of the process being manipulated.

New education system for construction of optical holography setup - Tangible learning with Augmented Reality

NASA Astrophysics Data System (ADS)

Yamaguchi, Takeshi; Yoshikawa, Hiroshi

2013-02-01

In case of teaching optical system construction, it is difficult to prepare the optical components for the attendance student. However the tangible learning is very important to master the optical system construction. It helps learners understand easily to use an inexpensive learning system that provides optical experiments experiences. Therefore, we propose the new education system for construction of optical setup with the augmented reality. To use the augmented reality, the proposed system can simulate the optical system construction by the direct hand control. Also, this system only requires an inexpensive web camera, printed makers and a personal computer. Since this system does not require the darkroom and the expensive optical equipments, the learners can study anytime, anywhere when they want to do. In this paper, we developed the system that can teach the optical system construction of the Denisyuk hologram and 2-step transmission type hologram. For the tangible learning and the easy understanding, the proposed system displays the CG objects of the optical components on the markers which are controlled by the learner's hands. The proposed system does not only display the CG object, but also display the light beam which is controlled by the optical components. To display the light beam that is hard to be seen directly, the learners can confirm about what is happening by the own manipulation. For the construction of optical holography setup, we arrange a laser, mirrors, a PBS (polarizing beam splitter), lenses, a polarizer, half-wave plates, spatial filters, an optical power meter and a recording plate. After the construction, proposed system can check optical setup correctly. In comparison with the learners who only read a book, the learners who use the system can construct the optical holography setup more quickly and correctly.

Supervised learning of postural tasks in patients with poststroke hemiparesis, Parkinson's disease or cerebellar ataxia.

PubMed

Ioffe, M E; Ustinova, K I; Chernikova, L A; Kulikov, M A

2006-01-01

Supervised learning of different postural tasks in patients with lesions of the motor cortex or pyramidal system (poststroke hemiparesis: 20 patients), nigro-striatal system (Parkinson's disease: 33 patients) and cerebellum (spinocerebellar ataxia: 37 patients) was studied. A control group consisted of 13 healthy subjects. The subjects stood on a force platform and were trained to change the position of the center of pressure (CP) presented as a cursor on a monitor screen in front of the patient. Subjects were instructed to align the CP with the target and then move the target by shifting the CP in the indicated direction. Two different tasks were used. In "Balls", the target (a ball) position varied randomly, so the subject learned a general strategy of voluntary CP control. In "Bricks", the subject had to always move the target in a single direction (downward) from the top to the bottom of the screen, so that a precise postural coordination had to be learned. The training consisted of 10 sessions for each task. The number of correctly performed trials for a session (2 min for each task) was scored. The voluntary control of the CP position was initially impaired in all groups of patients in both tasks. In "Balls", there were no differences between the groups of the patients on the first day. The learning course was somewhat better in hemiparetic patients than in the other groups. In "Bricks", the initial deficit was greater in the groups of parkinsonian and cerebellar patients than in hemiparetic patients. However, learning was more efficient in parkinsonian than in hemiparetic and cerebellar patients. After 10 days of training, the hemiparetic and cerebellar patients completed the acquisition at a certain level whereas the parkinsonian patients showed the ability for further improvement. The results suggest that motor cortex, cerebellum, and basal ganglia are involved in voluntary control of posture and learning different postural tasks. However, these structures play different roles in postural control and learning: basal ganglia are mainly involved in learning a general strategy of CP control while the function of the motor cortex chiefly concerns learning a specific CP trajectory. The cerebellum is involved in both kinds of learning.

[Development of a multimedia learning DM diet education program using standardized patients and analysis of its effects on clinical competency and learning satisfaction for nursing students].

PubMed

Hyun, Kyung Sun; Kang, Hyun Sook; Kim, Won Ock; Park, Sunhee; Lee, Jia; Sok, Sohyune

2009-04-01

The purpose of this study was to develop a multimedia learning program for patients with diabetes mellitus (DM) diet education using standardized patients and to examine the effects of the program on educational skills, communication skills, DM diet knowledge and learning satisfaction. The study employed a randomized control posttest non-synchronized design. The participants were 108 third year nursing students (52 experimental group, 56 control group) at K university in Seoul, Korea. The experimental group had regular lectures and the multimedia learning program for DM diet education using standardized patients while the control group had regular lectures only. The DM educational skills were measured by trained research assistants. The students who received the multimedia learning program scored higher for DM diet educational skills, communication skills and DM diet knowledge compared to the control group. Learning satisfaction of the experimental group was higher than the control group, but statistically insignificant. Clinical competency was improved for students receiving the multimedia learning program for DM diet education using standardized patients, but there was no statistically significant effect on learning satisfaction. In the nursing education system there is a need to develop and apply more multimedia materials for education and to use standardized patients effectively.

Fidelity-Based Ant Colony Algorithm with Q-learning of Quantum System

NASA Astrophysics Data System (ADS)

Liao, Qin; Guo, Ying; Tu, Yifeng; Zhang, Hang

2018-03-01

Quantum ant colony algorithm (ACA) has potential applications in quantum information processing, such as solutions of traveling salesman problem, zero-one knapsack problem, robot route planning problem, and so on. To shorten the search time of the ACA, we suggest the fidelity-based ant colony algorithm (FACA) for the control of quantum system. Motivated by structure of the Q-learning algorithm, we demonstrate the combination of a FACA with the Q-learning algorithm and suggest the design of a fidelity-based ant colony algorithm with the Q-learning to improve the performance of the FACA in a spin-1/2 quantum system. The numeric simulation results show that the FACA with the Q-learning can efficiently avoid trapping into local optimal policies and increase the speed of convergence process of quantum system.

Fidelity-Based Ant Colony Algorithm with Q-learning of Quantum System

NASA Astrophysics Data System (ADS)

Liao, Qin; Guo, Ying; Tu, Yifeng; Zhang, Hang

2017-12-01

Quantum ant colony algorithm (ACA) has potential applications in quantum information processing, such as solutions of traveling salesman problem, zero-one knapsack problem, robot route planning problem, and so on. To shorten the search time of the ACA, we suggest the fidelity-based ant colony algorithm (FACA) for the control of quantum system. Motivated by structure of the Q-learning algorithm, we demonstrate the combination of a FACA with the Q-learning algorithm and suggest the design of a fidelity-based ant colony algorithm with the Q-learning to improve the performance of the FACA in a spin-1/2 quantum system. The numeric simulation results show that the FACA with the Q-learning can efficiently avoid trapping into local optimal policies and increase the speed of convergence process of quantum system.

Enhancing Student Motivation and Learning within Adaptive Tutors

ERIC Educational Resources Information Center

Ostrow, Korinn S.

2015-01-01

My research is rooted in improving K-12 educational practice using motivational facets made possible through adaptive tutoring systems. In an attempt to isolate best practices within the science of learning, I conduct randomized controlled trials within ASSISTments, an online adaptive tutoring system that provides assistance and assessment to…

Robust iterative learning contouring controller with disturbance observer for machine tool feed drives.

PubMed

Simba, Kenneth Renny; Bui, Ba Dinh; Msukwa, Mathew Renny; Uchiyama, Naoki

2018-04-01

In feed drive systems, particularly machine tools, a contour error is more significant than the individual axial tracking errors from the view point of enhancing precision in manufacturing and production systems. The contour error must be within the permissible tolerance of given products. In machining complex or sharp-corner products, large contour errors occur mainly owing to discontinuous trajectories and the existence of nonlinear uncertainties. Therefore, it is indispensable to design robust controllers that can enhance the tracking ability of feed drive systems. In this study, an iterative learning contouring controller consisting of a classical Proportional-Derivative (PD) controller and disturbance observer is proposed. The proposed controller was evaluated experimentally by using a typical sharp-corner trajectory, and its performance was compared with that of conventional controllers. The results revealed that the maximum contour error can be reduced by about 37% on average. Copyright © 2018 ISA. Published by Elsevier Ltd. All rights reserved.

A Laboratory Testbed for Embedded Fuzzy Control

ERIC Educational Resources Information Center

Srivastava, S.; Sukumar, V.; Bhasin, P. S.; Arun Kumar, D.

2011-01-01

This paper presents a novel scheme called "Laboratory Testbed for Embedded Fuzzy Control of a Real Time Nonlinear System." The idea is based upon the fact that project-based learning motivates students to learn actively and to use their engineering skills acquired in their previous years of study. It also fosters initiative and focuses…

Motor-Skill Learning in an Insect Inspired Neuro-Computational Control System

PubMed Central

Arena, Eleonora; Arena, Paolo; Strauss, Roland; Patané, Luca

2017-01-01

In nature, insects show impressive adaptation and learning capabilities. The proposed computational model takes inspiration from specific structures of the insect brain: after proposing key hypotheses on the direct involvement of the mushroom bodies (MBs) and on their neural organization, we developed a new architecture for motor learning to be applied in insect-like walking robots. The proposed model is a nonlinear control system based on spiking neurons. MBs are modeled as a nonlinear recurrent spiking neural network (SNN) with novel characteristics, able to memorize time evolutions of key parameters of the neural motor controller, so that existing motor primitives can be improved. The adopted control scheme enables the structure to efficiently cope with goal-oriented behavioral motor tasks. Here, a six-legged structure, showing a steady-state exponentially stable locomotion pattern, is exposed to the need of learning new motor skills: moving through the environment, the structure is able to modulate motor commands and implements an obstacle climbing procedure. Experimental results on a simulated hexapod robot are reported; they are obtained in a dynamic simulation environment and the robot mimicks the structures of Drosophila melanogaster. PMID:28337138

Efficient retrieval of landscape Hessian: Forced optimal covariance adaptive learning

NASA Astrophysics Data System (ADS)

Shir, Ofer M.; Roslund, Jonathan; Whitley, Darrell; Rabitz, Herschel

2014-06-01

Knowledge of the Hessian matrix at the landscape optimum of a controlled physical observable offers valuable information about the system robustness to control noise. The Hessian can also assist in physical landscape characterization, which is of particular interest in quantum system control experiments. The recently developed landscape theoretical analysis motivated the compilation of an automated method to learn the Hessian matrix about the global optimum without derivative measurements from noisy data. The current study introduces the forced optimal covariance adaptive learning (FOCAL) technique for this purpose. FOCAL relies on the covariance matrix adaptation evolution strategy (CMA-ES) that exploits covariance information amongst the control variables by means of principal component analysis. The FOCAL technique is designed to operate with experimental optimization, generally involving continuous high-dimensional search landscapes (≳30) with large Hessian condition numbers (≳104). This paper introduces the theoretical foundations of the inverse relationship between the covariance learned by the evolution strategy and the actual Hessian matrix of the landscape. FOCAL is presented and demonstrated to retrieve the Hessian matrix with high fidelity on both model landscapes and quantum control experiments, which are observed to possess nonseparable, nonquadratic search landscapes. The recovered Hessian forms were corroborated by physical knowledge of the systems. The implications of FOCAL extend beyond the investigated studies to potentially cover other physically motivated multivariate landscapes.

Decentralized stabilization for a class of continuous-time nonlinear interconnected systems using online learning optimal control approach.

PubMed

Liu, Derong; Wang, Ding; Li, Hongliang

2014-02-01

In this paper, using a neural-network-based online learning optimal control approach, a novel decentralized control strategy is developed to stabilize a class of continuous-time nonlinear interconnected large-scale systems. First, optimal controllers of the isolated subsystems are designed with cost functions reflecting the bounds of interconnections. Then, it is proven that the decentralized control strategy of the overall system can be established by adding appropriate feedback gains to the optimal control policies of the isolated subsystems. Next, an online policy iteration algorithm is presented to solve the Hamilton-Jacobi-Bellman equations related to the optimal control problem. Through constructing a set of critic neural networks, the cost functions can be obtained approximately, followed by the control policies. Furthermore, the dynamics of the estimation errors of the critic networks are verified to be uniformly and ultimately bounded. Finally, a simulation example is provided to illustrate the effectiveness of the present decentralized control scheme.

Development and Flight Testing of a Neural Network Based Flight Control System on the NF-15B Aircraft

NASA Technical Reports Server (NTRS)

Bomben, Craig R.; Smolka, James W.; Bosworth, John T.; Silliams-Hayes, Peggy S.; Burken, John J.; Larson, Richard R.; Buschbacher, Mark J.; Maliska, Heather A.

2006-01-01

The Intelligent Flight Control System (IFCS) project at the NASA Dryden Flight Research Center, Edwards AFB, CA, has been investigating the use of neural network based adaptive control on a unique NF-15B test aircraft. The IFCS neural network is a software processor that stores measured aircraft response information to dynamically alter flight control gains. In 2006, the neural network was engaged and allowed to learn in real time to dynamically alter the aircraft handling qualities characteristics in the presence of actual aerodynamic failure conditions injected into the aircraft through the flight control system. The use of neural network and similar adaptive technologies in the design of highly fault and damage tolerant flight control systems shows promise in making future aircraft far more survivable than current technology allows. This paper will present the results of the IFCS flight test program conducted at the NASA Dryden Flight Research Center in 2006, with emphasis on challenges encountered and lessons learned.

Development of adaptive control applied to chaotic systems

NASA Astrophysics Data System (ADS)

Rhode, Martin Andreas

1997-12-01

Continuous-time derivative control and adaptive map-based recursive feedback control techniques are used to control chaos in a variety of systems and in situations that are of practical interest. The theoretical part of the research includes the review of fundamental concept of control theory in the context of its applications to deterministic chaotic systems, the development of a new adaptive algorithm to identify the linear system properties necessary for control, and the extension of the recursive proportional feedback control technique, RPF, to high dimensional systems. Chaos control was applied to models of a thermal pulsed combustor, electro-chemical dissolution and the hyperchaotic Rossler system. Important implications for combustion engineering were suggested by successful control of the model of the thermal pulsed combustor. The system was automatically tracked while maintaining control into regions of parameter and state space where no stable attractors exist. In a simulation of the electrochemical dissolution system, application of derivative control to stabilize a steady state, and adaptive RPF to stabilize a period one orbit, was demonstrated. The high dimensional adaptive control algorithm was applied in a simulation using the Rossler hyperchaotic system, where a period-two orbit with two unstable directions was stabilized and tracked over a wide range of a system parameter. In the experimental part, the electrochemical system was studied in parameter space, by scanning the applied potential and the frequency of the rotating copper disk. The automated control algorithm is demonstrated to be effective when applied to stabilize a period-one orbit in the experiment. We show the necessity of small random perturbations applied to the system in order to both learn the dynamics and control the system at the same time. The simultaneous learning and control capability is shown to be an important part of the active feedback control.

Towards Greater Learner Control: Web Supported Project-Based Learning

ERIC Educational Resources Information Center

Guthrie, Cameron

2010-01-01

Project-based learning has been suggested as an appropriate pedagogy to prepare students in information systems for the realities of the business world. Web-based resources have been used to support such pedagogy with mixed results. The paper argues that the design of web-based learning support to cater to different learning styles may give…

Web-based e-learning and virtual lab of human-artificial immune system.

PubMed

Gong, Tao; Ding, Yongsheng; Xiong, Qin

2014-05-01

Human immune system is as important in keeping the body healthy as the brain in supporting the intelligence. However, the traditional models of the human immune system are built on the mathematics equations, which are not easy for students to understand. To help the students to understand the immune systems, a web-based e-learning approach with virtual lab is designed for the intelligent system control course by using new intelligent educational technology. Comparing the traditional graduate educational model within the classroom, the web-based e-learning with the virtual lab shows the higher inspiration in guiding the graduate students to think independently and innovatively, as the students said. It has been found that this web-based immune e-learning system with the online virtual lab is useful for teaching the graduate students to understand the immune systems in an easier way and design their simulations more creatively and cooperatively. The teaching practice shows that the optimum web-based e-learning system can be used to increase the learning effectiveness of the students.

Biomimetic Hybrid Feedback Feedforward Neural-Network Learning Control.

PubMed

Pan, Yongping; Yu, Haoyong

2017-06-01

This brief presents a biomimetic hybrid feedback feedforward neural-network learning control (NNLC) strategy inspired by the human motor learning control mechanism for a class of uncertain nonlinear systems. The control structure includes a proportional-derivative controller acting as a feedback servo machine and a radial-basis-function (RBF) NN acting as a feedforward predictive machine. Under the sufficient constraints on control parameters, the closed-loop system achieves semiglobal practical exponential stability, such that an accurate NN approximation is guaranteed in a local region along recurrent reference trajectories. Compared with the existing NNLC methods, the novelties of the proposed method include: 1) the implementation of an adaptive NN control to guarantee plant states being recurrent is not needed, since recurrent reference signals rather than plant states are utilized as NN inputs, which greatly simplifies the analysis and synthesis of the NNLC and 2) the domain of NN approximation can be determined a priori by the given reference signals, which leads to an easy construction of the RBF-NNs. Simulation results have verified the effectiveness of this approach.

Research on cultivating medical students' self-learning ability using teaching system integrated with learning analysis technology.

PubMed

Luo, Hong; Wu, Cheng; He, Qian; Wang, Shi-Yong; Ma, Xiu-Qiang; Wang, Ri; Li, Bing; He, Jia

2015-01-01

Along with the advancement of information technology and the era of big data education, using learning process data to provide strategic decision-making in cultivating and improving medical students' self-learning ability has become a trend in educational research. Educator Abuwen Toffler said once, the illiterates in the future may not be the people not able to read and write, but not capable to know how to learn. Serving as educational institutions cultivating medical students' learning ability, colleges and universities should not only instruct specific professional knowledge and skills, but also develop medical students' self-learning ability. In this research, we built a teaching system which can help to restore medical students' self-learning processes and analyze their learning outcomes and behaviors. To evaluate the effectiveness of the system in supporting medical students' self-learning, an experiment was conducted in 116 medical students from two grades. The results indicated that problems in self-learning process through this system was consistent with problems raised from traditional classroom teaching. Moreover, the experimental group (using this system) acted better than control group (using traditional classroom teaching) to some extent. Thus, this system can not only help medical students to develop their self-learning ability, but also enhances the ability of teachers to target medical students' questions quickly, improving the efficiency of answering questions in class.

Tryptophan Depletion Promotes Habitual over Goal-Directed Control of Appetitive Responding in Humans.

PubMed

Worbe, Yulia; Savulich, George; de Wit, Sanne; Fernandez-Egea, Emilio; Robbins, Trevor W

2015-02-05

Optimal behavioral performance results from a balance between goal-directed and habitual systems of behavioral control, which are modulated by ascending monoaminergic projections. While the role of the dopaminergic system in behavioral control has been recently addressed, the extent to which changes in global serotonin neurotransmission could influence these 2 systems is still poorly understood. We employed the dietary acute tryptophan depletion procedure to reduce serotonin neurotransmission in 18 healthy volunteers and 18 matched controls. We used a 3-stage instrumental learning paradigm that includes an initial instrumental learning stage, a subsequent outcome-devaluation test, and a slip-of-action stage, which directly tests the balance between hypothetical goal-directed and habitual systems. We also employed a separate response inhibition control test to assess the behavioral specificity of the results. Acute tryptophan depletion produced a shift of behavioral performance towards habitual responding as indexed by performance on the slip-of-action test. Moreover, greater habitual responding in the acute tryptophan depletion group was predicted by a steeper decline in plasma tryptophan levels. In contrast, acute tryptophan depletion left intact the ability to use discriminative stimuli to guide instrumental choice as indexed by the instrumental learning stage and did not impair inhibitory response control. The major implication of this study is that serotonin modulates the balance between goal-directed and stimulus-response habitual systems of behavioral control. Our findings thus imply that diminished serotonin neurotransmission shifts behavioral control towards habitual responding. © The Author 2015. Published by Oxford University Press on behalf of CINP.

Enriching Adaptation in E-Learning Systems through a Situation-Aware Ontology Network

ERIC Educational Resources Information Center

Pernas, Ana Marilza; Diaz, Alicia; Motz, Regina; de Oliveira, Jose Palazzo Moreira

2012-01-01

Purpose: The broader adoption of the internet along with web-based systems has defined a new way of exchanging information. That advance added by the multiplication of mobile devices has required systems to be even more flexible and personalized. Maybe because of that, the traditional teaching-controlled learning style has given up space to a new…

Beyond adaptive-critic creative learning for intelligent mobile robots

NASA Astrophysics Data System (ADS)

Liao, Xiaoqun; Cao, Ming; Hall, Ernest L.

2001-10-01

Intelligent industrial and mobile robots may be considered proven technology in structured environments. Teach programming and supervised learning methods permit solutions to a variety of applications. However, we believe that to extend the operation of these machines to more unstructured environments requires a new learning method. Both unsupervised learning and reinforcement learning are potential candidates for these new tasks. The adaptive critic method has been shown to provide useful approximations or even optimal control policies to non-linear systems. The purpose of this paper is to explore the use of new learning methods that goes beyond the adaptive critic method for unstructured environments. The adaptive critic is a form of reinforcement learning. A critic element provides only high level grading corrections to a cognition module that controls the action module. In the proposed system the critic's grades are modeled and forecasted, so that an anticipated set of sub-grades are available to the cognition model. The forecasting grades are interpolated and are available on the time scale needed by the action model. The success of the system is highly dependent on the accuracy of the forecasted grades and adaptability of the action module. Examples from the guidance of a mobile robot are provided to illustrate the method for simple line following and for the more complex navigation and control in an unstructured environment. The theory presented that is beyond the adaptive critic may be called creative theory. Creative theory is a form of learning that models the highest level of human learning - imagination. The application of the creative theory appears to not only be to mobile robots but also to many other forms of human endeavor such as educational learning and business forecasting. Reinforcement learning such as the adaptive critic may be applied to known problems to aid in the discovery of their solutions. The significance of creative theory is that it permits the discovery of the unknown problems, ones that are not yet recognized but may be critical to survival or success.

Unsupervised learning in general connectionist systems.

PubMed

Dente, J A; Mendes, R Vilela

1996-01-01

There is a common framework in which different connectionist systems may be treated in a unified way. The general system in which they may all be mapped is a network which, in addition to the connection strengths, has an adaptive node parameter controlling the output intensity. In this paper we generalize two neural network learning schemes to networks with node parameters. In generalized Hebbian learning we find improvements to the convergence rate for small eigenvalues in principal component analysis. For competitive learning the use of node parameters also seems useful in that, by emphasizing or de-emphasizing the dominance of winning neurons, either improved robustness or discrimination is obtained.

Online Learning Flight Control for Intelligent Flight Control Systems (IFCS)

NASA Technical Reports Server (NTRS)

Niewoehner, Kevin R.; Carter, John (Technical Monitor)

2001-01-01

The research accomplishments for the cooperative agreement 'Online Learning Flight Control for Intelligent Flight Control Systems (IFCS)' include the following: (1) previous IFC program data collection and analysis; (2) IFC program support site (configured IFC systems support network, configured Tornado/VxWorks OS development system, made Configuration and Documentation Management Systems Internet accessible); (3) Airborne Research Test Systems (ARTS) II Hardware (developed hardware requirements specification, developing environmental testing requirements, hardware design, and hardware design development); (4) ARTS II software development laboratory unit (procurement of lab style hardware, configured lab style hardware, and designed interface module equivalent to ARTS II faceplate); (5) program support documentation (developed software development plan, configuration management plan, and software verification and validation plan); (6) LWR algorithm analysis (performed timing and profiling on algorithm); (7) pre-trained neural network analysis; (8) Dynamic Cell Structures (DCS) Neural Network Analysis (performing timing and profiling on algorithm); and (9) conducted technical interchange and quarterly meetings to define IFC research goals.

Vision Based Autonomous Robotic Control for Advanced Inspection and Repair

NASA Technical Reports Server (NTRS)

Wehner, Walter S.

2014-01-01

The advanced inspection system is an autonomous control and analysis system that improves the inspection and remediation operations for ground and surface systems. It uses optical imaging technology with intelligent computer vision algorithms to analyze physical features of the real-world environment to make decisions and learn from experience. The advanced inspection system plans to control a robotic manipulator arm, an unmanned ground vehicle and cameras remotely, automatically and autonomously. There are many computer vision, image processing and machine learning techniques available as open source for using vision as a sensory feedback in decision-making and autonomous robotic movement. My responsibilities for the advanced inspection system are to create a software architecture that integrates and provides a framework for all the different subsystem components; identify open-source algorithms and techniques; and integrate robot hardware.

Autonomous Energy Grids | Grid Modernization | NREL

Science.gov Websites

control themselves using advanced machine learning and simulation to create resilient, reliable, and affordable optimized energy systems. Current frameworks to monitor, control, and optimize large-scale energy of optimization theory, control theory, big data analytics, and complex system theory and modeling to

Neural network-based model reference adaptive control system.

PubMed

Patino, H D; Liu, D

2000-01-01

In this paper, an approach to model reference adaptive control based on neural networks is proposed and analyzed for a class of first-order continuous-time nonlinear dynamical systems. The controller structure can employ either a radial basis function network or a feedforward neural network to compensate adaptively the nonlinearities in the plant. A stable controller-parameter adjustment mechanism, which is determined using the Lyapunov theory, is constructed using a sigma-modification-type updating law. The evaluation of control error in terms of the neural network learning error is performed. That is, the control error converges asymptotically to a neighborhood of zero, whose size is evaluated and depends on the approximation error of the neural network. In the design and analysis of neural network-based control systems, it is important to take into account the neural network learning error and its influence on the control error of the plant. Simulation results showing the feasibility and performance of the proposed approach are given.

The control of tonic pain by active relief learning

PubMed Central

Mano, Hiroaki; Lee, Michael; Yoshida, Wako; Kawato, Mitsuo; Robbins, Trevor W

2018-01-01

Tonic pain after injury characterises a behavioural state that prioritises recovery. Although generally suppressing cognition and attention, tonic pain needs to allow effective relief learning to reduce the cause of the pain. Here, we describe a central learning circuit that supports learning of relief and concurrently suppresses the level of ongoing pain. We used computational modelling of behavioural, physiological and neuroimaging data in two experiments in which subjects learned to terminate tonic pain in static and dynamic escape-learning paradigms. In both studies, we show that active relief-seeking involves a reinforcement learning process manifest by error signals observed in the dorsal putamen. Critically, this system uses an uncertainty (‘associability’) signal detected in pregenual anterior cingulate cortex that both controls the relief learning rate, and endogenously and parametrically modulates the level of tonic pain. The results define a self-organising learning circuit that reduces ongoing pain when learning about potential relief. PMID:29482716

Fuzzy Logic Based Anomaly Detection for Embedded Network Security Cyber Sensor

DOE Office of Scientific and Technical Information (OSTI.GOV)

Ondrej Linda; Todd Vollmer; Jason Wright

Resiliency and security in critical infrastructure control systems in the modern world of cyber terrorism constitute a relevant concern. Developing a network security system specifically tailored to the requirements of such critical assets is of a primary importance. This paper proposes a novel learning algorithm for anomaly based network security cyber sensor together with its hardware implementation. The presented learning algorithm constructs a fuzzy logic rule based model of normal network behavior. Individual fuzzy rules are extracted directly from the stream of incoming packets using an online clustering algorithm. This learning algorithm was specifically developed to comply with the constrainedmore » computational requirements of low-cost embedded network security cyber sensors. The performance of the system was evaluated on a set of network data recorded from an experimental test-bed mimicking the environment of a critical infrastructure control system.« less

Multiresolutional schemata for unsupervised learning of autonomous robots for 3D space operation

NASA Technical Reports Server (NTRS)

Lacaze, Alberto; Meystel, Michael; Meystel, Alex

1994-01-01

This paper describes a novel approach to the development of a learning control system for autonomous space robot (ASR) which presents the ASR as a 'baby' -- that is, a system with no a priori knowledge of the world in which it operates, but with behavior acquisition techniques that allows it to build this knowledge from the experiences of actions within a particular environment (we will call it an Astro-baby). The learning techniques are rooted in the recursive algorithm for inductive generation of nested schemata molded from processes of early cognitive development in humans. The algorithm extracts data from the environment and by means of correlation and abduction, it creates schemata that are used for control. This system is robust enough to deal with a constantly changing environment because such changes provoke the creation of new schemata by generalizing from experiences, while still maintaining minimal computational complexity, thanks to the system's multiresolutional nature.

Adaptive Performance Seeking Control Using Fuzzy Model Reference Learning Control and Positive Gradient Control

NASA Technical Reports Server (NTRS)

Kopasakis, George

1997-01-01

Performance Seeking Control attempts to find the operating condition that will generate optimal performance and control the plant at that operating condition. In this paper a nonlinear multivariable Adaptive Performance Seeking Control (APSC) methodology will be developed and it will be demonstrated on a nonlinear system. The APSC is comprised of the Positive Gradient Control (PGC) and the Fuzzy Model Reference Learning Control (FMRLC). The PGC computes the positive gradients of the desired performance function with respect to the control inputs in order to drive the plant set points to the operating point that will produce optimal performance. The PGC approach will be derived in this paper. The feedback control of the plant is performed by the FMRLC. For the FMRLC, the conventional fuzzy model reference learning control methodology is utilized, with guidelines generated here for the effective tuning of the FMRLC controller.

An fMRI investigation of the fronto-striatal learning system in women who exhibit eating disorder behaviors

PubMed Central

Celone, Kim A.; Thompson-Brenner, Heather; Ross, Robert S.; Pratt, Elizabeth M.; Stern, Chantal E.

2013-01-01

In the present study, we sought to examine whether the fronto-striatal learning system, which has been implicated in bulimia nervosa, would demonstrate altered BOLD activity during probabilistic category learning in women who met subthreshold criteria for bulimia nervosa (Sub-BN). Sub-BN, which falls within the clinical category of Eating Disorder Not Otherwise Specified (EDNOS), is comprised of individuals who demonstrate recurrent binge eating, efforts to minimize their caloric intake and caloric retention, and elevated levels of concern about shape, weight, and/or eating, but just fail to meet the diagnostic threshold for bulimia nervosa (BN). fMRI data were collected from eighteen women with subthreshold-BN (Sub-BN) and nineteen healthy control women group-matched for age, education and body mass index (MC) during the weather prediction task. Sub-BN participants demonstrated increased caudate nucleus and dorsolateral prefrontal cortex (DLPFC) activation during the learning of probabilistic categories. Though the two subject groups did not differ in behavioral performance, over the course of learning, Sub-BN participants showed a dynamic pattern of brain activity differences when compared to matched control participants. Regions implicated in episodic memory, including the medial temporal lobe (MTL), retrosplenial cortex, middle frontal gyrus, and anterior and posterior cingulate cortex showed decreased activity in the Sub-BN participants compared to MCs during early learning which was followed by increased involvement of the DLPFC during later learning. These findings demonstrate that women with Sub-BN demonstrate differences in fronto-striatal learning system activity, as well as a distinct functional pattern between fronto-striatal and MTL learning systems during the course of implicit probabilistic category learning. PMID:21419229

DOE Office of Scientific and Technical Information (OSTI.GOV)

Beaver, Justin M; Borges, Raymond Charles; Buckner, Mark A

Critical infrastructure Supervisory Control and Data Acquisition (SCADA) systems were designed to operate on closed, proprietary networks where a malicious insider posed the greatest threat potential. The centralization of control and the movement towards open systems and standards has improved the efficiency of industrial control, but has also exposed legacy SCADA systems to security threats that they were not designed to mitigate. This work explores the viability of machine learning methods in detecting the new threat scenarios of command and data injection. Similar to network intrusion detection systems in the cyber security domain, the command and control communications in amore » critical infrastructure setting are monitored, and vetted against examples of benign and malicious command traffic, in order to identify potential attack events. Multiple learning methods are evaluated using a dataset of Remote Terminal Unit communications, which included both normal operations and instances of command and data injection attack scenarios.« less

Simulating closed- and open-loop voluntary movement: a nonlinear control-systems approach.

PubMed

Davidson, Paul R; Jones, Richard D; Andreae, John H; Sirisena, Harsha R

2002-11-01

In many recent human motor control models, including feedback-error learning and adaptive model theory (AMT), feedback control is used to correct errors while an inverse model is simultaneously tuned to provide accurate feedforward control. This popular and appealing hypothesis, based on a combination of psychophysical observations and engineering considerations, predicts that once the tuning of the inverse model is complete the role of feedback control is limited to the correction of disturbances. This hypothesis was tested by looking at the open-loop behavior of the human motor system during adaptation. An experiment was carried out involving 20 normal adult subjects who learned a novel visuomotor relationship on a pursuit tracking task with a steering wheel for input. During learning, the response cursor was periodically blanked, removing all feedback about the external system (i.e., about the relationship between hand motion and response cursor motion). Open-loop behavior was not consistent with a progressive transfer from closed- to open-loop control. Our recently developed computational model of the brain--a novel nonlinear implementation of AMT--was able to reproduce the observed closed- and open-loop results. In contrast, other control-systems models exhibited only minimal feedback control following adaptation, leading to incorrect open-loop behavior. This is because our model continues to use feedback to control slow movements after adaptation is complete. This behavior enhances the internal stability of the inverse model. In summary, our computational model is currently the only motor control model able to accurately simulate the closed- and open-loop characteristics of the experimental response trajectories.

Ryan King | NREL

Science.gov Websites

research focuses on optimization and machine learning applied to complex energy systems and turbulent flows techniques to improve wind plant design and controls and developed a new data-driven machine learning closure

Open Integrated Personal Learning Environment: Towards a New Conception of the ICT-Based Learning Processes

NASA Astrophysics Data System (ADS)

Conde, Miguel Ángel; García-Peñalvo, Francisco José; Casany, Marià José; Alier Forment, Marc

Learning processes are changing related to technological and sociological evolution, taking this in to account, a new learning strategy must be considered. Specifically what is needed is to give an effective step towards the eLearning 2.0 environments consolidation. This must imply the fusion of the advantages of the traditional LMS (Learning Management System) - more formative program control and planning oriented - with the social learning and the flexibility of the web 2.0 educative applications.

Discrete-time online learning control for a class of unknown nonaffine nonlinear systems using reinforcement learning.

PubMed

Yang, Xiong; Liu, Derong; Wang, Ding; Wei, Qinglai

2014-07-01

In this paper, a reinforcement-learning-based direct adaptive control is developed to deliver a desired tracking performance for a class of discrete-time (DT) nonlinear systems with unknown bounded disturbances. We investigate multi-input-multi-output unknown nonaffine nonlinear DT systems and employ two neural networks (NNs). By using Implicit Function Theorem, an action NN is used to generate the control signal and it is also designed to cancel the nonlinearity of unknown DT systems, for purpose of utilizing feedback linearization methods. On the other hand, a critic NN is applied to estimate the cost function, which satisfies the recursive equations derived from heuristic dynamic programming. The weights of both the action NN and the critic NN are directly updated online instead of offline training. By utilizing Lyapunov's direct method, the closed-loop tracking errors and the NN estimated weights are demonstrated to be uniformly ultimately bounded. Two numerical examples are provided to show the effectiveness of the present approach. Copyright © 2014 Elsevier Ltd. All rights reserved.

A Matlab/Simulink-Based Interactive Module for Servo Systems Learning

ERIC Educational Resources Information Center

Aliane, N.

2010-01-01

This paper presents an interactive module for learning both the fundamental and practical issues of servo systems. This module, developed using Simulink in conjunction with the Matlab graphical user interface (Matlab-GUI) tool, is used to supplement conventional lectures in control engineering and robotics subjects. First, the paper introduces the…

Adaptive filter design using recurrent cerebellar model articulation controller.

PubMed

Lin, Chih-Min; Chen, Li-Yang; Yeung, Daniel S

2010-07-01

A novel adaptive filter is proposed using a recurrent cerebellar-model-articulation-controller (CMAC). The proposed locally recurrent globally feedforward recurrent CMAC (RCMAC) has favorable properties of small size, good generalization, rapid learning, and dynamic response, thus it is more suitable for high-speed signal processing. To provide fast training, an efficient parameter learning algorithm based on the normalized gradient descent method is presented, in which the learning rates are on-line adapted. Then the Lyapunov function is utilized to derive the conditions of the adaptive learning rates, so the stability of the filtering error can be guaranteed. To demonstrate the performance of the proposed adaptive RCMAC filter, it is applied to a nonlinear channel equalization system and an adaptive noise cancelation system. The advantages of the proposed filter over other adaptive filters are verified through simulations.

Design and Control of Large Collections of Learning Agents

NASA Technical Reports Server (NTRS)

Agogino, Adrian

2001-01-01

The intelligent control of multiple autonomous agents is an important yet difficult task. Previous methods used to address this problem have proved to be either too brittle, too hard to use, or not scalable to large systems. The 'Collective Intelligence' project at NASA/Ames provides an elegant, machine-learning approach to address these problems. This approach mathematically defines some essential properties that a reward system should have to promote coordinated behavior among reinforcement learners. This work has focused on creating additional key properties and algorithms within the mathematics of the Collective Intelligence framework. One of the additions will allow agents to learn more quickly, in a more coordinated manner. The other will let agents learn with less knowledge of their environment. These additions will allow the framework to be applied more easily, to a much larger domain of multi-agent problems.

Case Study: Test Results of a Tool and Method for In-Flight, Adaptive Control System Verification on a NASA F-15 Flight Research Aircraft

NASA Technical Reports Server (NTRS)

Jacklin, Stephen A.; Schumann, Johann; Guenther, Kurt; Bosworth, John

2006-01-01

Adaptive control technologies that incorporate learning algorithms have been proposed to enable autonomous flight control and to maintain vehicle performance in the face of unknown, changing, or poorly defined operating environments [1-2]. At the present time, however, it is unknown how adaptive algorithms can be routinely verified, validated, and certified for use in safety-critical applications. Rigorous methods for adaptive software verification end validation must be developed to ensure that. the control software functions as required and is highly safe and reliable. A large gap appears to exist between the point at which control system designers feel the verification process is complete, and when FAA certification officials agree it is complete. Certification of adaptive flight control software verification is complicated by the use of learning algorithms (e.g., neural networks) and degrees of system non-determinism. Of course, analytical efforts must be made in the verification process to place guarantees on learning algorithm stability, rate of convergence, and convergence accuracy. However, to satisfy FAA certification requirements, it must be demonstrated that the adaptive flight control system is also able to fail and still allow the aircraft to be flown safely or to land, while at the same time providing a means of crew notification of the (impending) failure. It was for this purpose that the NASA Ames Confidence Tool was developed [3]. This paper presents the Confidence Tool as a means of providing in-flight software assurance monitoring of an adaptive flight control system. The paper will present the data obtained from flight testing the tool on a specially modified F-15 aircraft designed to simulate loss of flight control faces.

Examining the Effect of Time Constraint on the Online Mastery Learning Approach towards Improving Postgraduate Students' Achievement

ERIC Educational Resources Information Center

Ee, Mong Shan; Yeoh, William; Boo, Yee Ling; Boulter, Terry

2018-01-01

Time control plays a critical role within the online mastery learning (OML) approach. This paper examines the two commonly implemented mastery learning strategies--personalised system of instructions and learning for mastery (LFM)--by focusing on what occurs when there is an instructional time constraint. Using a large data set from a postgraduate…

Role of cerebellum in learning postural tasks.

PubMed

Ioffe, M E; Chernikova, L A; Ustinova, K I

2007-01-01

For a long time, the cerebellum has been known to be a structure related to posture and equilibrium control. According to the anatomic structure of inputs and internal structure of the cerebellum, its role in learning was theoretically reasoned and experimentally proved. The hypothesis of an inverse internal model based on feedback-error learning mechanism combines feedforward control by the cerebellum and feedback control by the cerebral motor cortex. The cerebellar cortex is suggested to acquire internal models of the body and objects in the external world. During learning of a new tool the motor cortex receives feedback from the realized movement while the cerebellum produces only feedforward command. To realize a desired movement without feedback of the realized movement, the cerebellum needs to form an inverse model of the hand/arm system. This suggestion was supported by FMRi data. The role of cerebellum in learning new postural tasks mainly concerns reorganization of natural synergies. A learned postural pattern in dogs has been shown to be disturbed after lesions of the cerebral motor cortex or cerebellar nuclei. In humans, learning voluntary control of center of pressure position is greatly disturbed after cerebellar lesions. However, motor cortex and basal ganglia are also involved in the feedback learning postural tasks.

Structure learning in action

PubMed Central

Braun, Daniel A.; Mehring, Carsten; Wolpert, Daniel M.

2010-01-01

‘Learning to learn’ phenomena have been widely investigated in cognition, perception and more recently also in action. During concept learning tasks, for example, it has been suggested that characteristic features are abstracted from a set of examples with the consequence that learning of similar tasks is facilitated—a process termed ‘learning to learn’. From a computational point of view such an extraction of invariants can be regarded as learning of an underlying structure. Here we review the evidence for structure learning as a ‘learning to learn’ mechanism, especially in sensorimotor control where the motor system has to adapt to variable environments. We review studies demonstrating that common features of variable environments are extracted during sensorimotor learning and exploited for efficient adaptation in novel tasks. We conclude that structure learning plays a fundamental role in skill learning and may underlie the unsurpassed flexibility and adaptability of the motor system. PMID:19720086

Neural dynamic programming and its application to control systems

NASA Astrophysics Data System (ADS)

Seong, Chang-Yun

There are few general practical feedback control methods for nonlinear MIMO (multi-input-multi-output) systems, although such methods exist for their linear counterparts. Neural Dynamic Programming (NDP) is proposed as a practical design method of optimal feedback controllers for nonlinear MIMO systems. NDP is an offspring of both neural networks and optimal control theory. In optimal control theory, the optimal solution to any nonlinear MIMO control problem may be obtained from the Hamilton-Jacobi-Bellman equation (HJB) or the Euler-Lagrange equations (EL). The two sets of equations provide the same solution in different forms: EL leads to a sequence of optimal control vectors, called Feedforward Optimal Control (FOC); HJB yields a nonlinear optimal feedback controller, called Dynamic Programming (DP). DP produces an optimal solution that can reject disturbances and uncertainties as a result of feedback. Unfortunately, computation and storage requirements associated with DP solutions can be problematic, especially for high-order nonlinear systems. This dissertation presents an approximate technique for solving the DP problem based on neural network techniques that provides many of the performance benefits (e.g., optimality and feedback) of DP and benefits from the numerical properties of neural networks. We formulate neural networks to approximate optimal feedback solutions whose existence DP justifies. We show the conditions under which NDP closely approximates the optimal solution. Finally, we introduce the learning operator characterizing the learning process of the neural network in searching the optimal solution. The analysis of the learning operator provides not only a fundamental understanding of the learning process in neural networks but also useful guidelines for selecting the number of weights of the neural network. As a result, NDP finds---with a reasonable amount of computation and storage---the optimal feedback solutions to nonlinear MIMO control problems that would be very difficult to solve with DP. NDP was demonstrated on several applications such as the lateral autopilot logic for a Boeing 747, the minimum fuel control of a double-integrator plant with bounded control, the backward steering of a two-trailer truck, and the set-point control of a two-link robot arm.

Intelligent control of an IPMC actuated manipulator using emotional learning-based controller

NASA Astrophysics Data System (ADS)

Shariati, Azadeh; Meghdari, Ali; Shariati, Parham

2008-08-01

In this research an intelligent emotional learning controller, Takagi- Sugeno- Kang (TSK) is applied to govern the dynamics of a novel Ionic-Polymer Metal Composite (IPMC) actuated manipulator. Ionic-Polymer Metal Composites are active actuators that show very large deformation in existence of low applied voltage. In this research, a new IPMC actuator is considered and applied to a 2-dof miniature manipulator. This manipulator is designed for miniature tasks. The control system consists of a set of neurofuzzy controller whose parameters are adapted according to the emotional learning rules, and a critic with task to assess the present situation resulted from the applied control action in terms of satisfactory achievement of the control goals and provides the emotional signal (the stress). The controller modifies its characteristics so that the critic's stress decreased.

Dynamical Motor Control Learned with Deep Deterministic Policy Gradient

PubMed Central

2018-01-01

Conventional models of motor control exploit the spatial representation of the controlled system to generate control commands. Typically, the control command is gained with the feedback state of a specific instant in time, which behaves like an optimal regulator or spatial filter to the feedback state. Yet, recent neuroscience studies found that the motor network may constitute an autonomous dynamical system and the temporal patterns of the control command can be contained in the dynamics of the motor network, that is, the dynamical system hypothesis (DSH). Inspired by these findings, here we propose a computational model that incorporates this neural mechanism, in which the control command could be unfolded from a dynamical controller whose initial state is specified with the task parameters. The model is trained in a trial-and-error manner in the framework of deep deterministic policy gradient (DDPG). The experimental results show that the dynamical controller successfully learns the control policy for arm reaching movements, while the analysis of the internal activities of the dynamical controller provides the computational evidence to the DSH of the neural coding in motor cortices. PMID:29666634

Dynamical Motor Control Learned with Deep Deterministic Policy Gradient.

PubMed

Shi, Haibo; Sun, Yaoru; Li, Jie

2018-01-01

Conventional models of motor control exploit the spatial representation of the controlled system to generate control commands. Typically, the control command is gained with the feedback state of a specific instant in time, which behaves like an optimal regulator or spatial filter to the feedback state. Yet, recent neuroscience studies found that the motor network may constitute an autonomous dynamical system and the temporal patterns of the control command can be contained in the dynamics of the motor network, that is, the dynamical system hypothesis (DSH). Inspired by these findings, here we propose a computational model that incorporates this neural mechanism, in which the control command could be unfolded from a dynamical controller whose initial state is specified with the task parameters. The model is trained in a trial-and-error manner in the framework of deep deterministic policy gradient (DDPG). The experimental results show that the dynamical controller successfully learns the control policy for arm reaching movements, while the analysis of the internal activities of the dynamical controller provides the computational evidence to the DSH of the neural coding in motor cortices.

Effective Information Extraction Framework for Heterogeneous Clinical Reports Using Online Machine Learning and Controlled Vocabularies.

PubMed

Zheng, Shuai; Lu, James J; Ghasemzadeh, Nima; Hayek, Salim S; Quyyumi, Arshed A; Wang, Fusheng

2017-05-09

Extracting structured data from narrated medical reports is challenged by the complexity of heterogeneous structures and vocabularies and often requires significant manual effort. Traditional machine-based approaches lack the capability to take user feedbacks for improving the extraction algorithm in real time. Our goal was to provide a generic information extraction framework that can support diverse clinical reports and enables a dynamic interaction between a human and a machine that produces highly accurate results. A clinical information extraction system IDEAL-X has been built on top of online machine learning. It processes one document at a time, and user interactions are recorded as feedbacks to update the learning model in real time. The updated model is used to predict values for extraction in subsequent documents. Once prediction accuracy reaches a user-acceptable threshold, the remaining documents may be batch processed. A customizable controlled vocabulary may be used to support extraction. Three datasets were used for experiments based on report styles: 100 cardiac catheterization procedure reports, 100 coronary angiographic reports, and 100 integrated reports-each combines history and physical report, discharge summary, outpatient clinic notes, outpatient clinic letter, and inpatient discharge medication report. Data extraction was performed by 3 methods: online machine learning, controlled vocabularies, and a combination of these. The system delivers results with F1 scores greater than 95%. IDEAL-X adopts a unique online machine learning-based approach combined with controlled vocabularies to support data extraction for clinical reports. The system can quickly learn and improve, thus it is highly adaptable. ©Shuai Zheng, James J Lu, Nima Ghasemzadeh, Salim S Hayek, Arshed A Quyyumi, Fusheng Wang. Originally published in JMIR Medical Informatics (http://medinform.jmir.org), 09.05.2017.

An intelligent agent for optimal river-reservoir system management

NASA Astrophysics Data System (ADS)

Rieker, Jeffrey D.; Labadie, John W.

2012-09-01

A generalized software package is presented for developing an intelligent agent for stochastic optimization of complex river-reservoir system management and operations. Reinforcement learning is an approach to artificial intelligence for developing a decision-making agent that learns the best operational policies without the need for explicit probabilistic models of hydrologic system behavior. The agent learns these strategies experientially in a Markov decision process through observational interaction with the environment and simulation of the river-reservoir system using well-calibrated models. The graphical user interface for the reinforcement learning process controller includes numerous learning method options and dynamic displays for visualizing the adaptive behavior of the agent. As a case study, the generalized reinforcement learning software is applied to developing an intelligent agent for optimal management of water stored in the Truckee river-reservoir system of California and Nevada for the purpose of streamflow augmentation for water quality enhancement. The intelligent agent successfully learns long-term reservoir operational policies that specifically focus on mitigating water temperature extremes during persistent drought periods that jeopardize the survival of threatened and endangered fish species.

Evaluation of an e-learning system for diagnosis of gastric lesions using magnifying narrow-band imaging: a multicenter randomized controlled study.

PubMed

Nakanishi, Hiroyoshi; Doyama, Hisashi; Ishikawa, Hideki; Uedo, Noriya; Gotoda, Takuji; Kato, Mototsugu; Nagao, Shigeaki; Nagami, Yasuaki; Aoyagi, Hiroyuki; Imagawa, Atsushi; Kodaira, Junichi; Mitsui, Shinya; Kobayashi, Nozomu; Muto, Manabu; Takatori, Hajime; Abe, Takashi; Tsujii, Masahiko; Watari, Jiro; Ishiyama, Shuhei; Oda, Ichiro; Ono, Hiroyuki; Kaneko, Kazuhiro; Yokoi, Chizu; Ueo, Tetsuya; Uchita, Kunihisa; Matsumoto, Kenshi; Kanesaka, Takashi; Morita, Yoshinori; Katsuki, Shinichi; Nishikawa, Jun; Inamura, Katsuhisa; Kinjo, Tetsu; Yamamoto, Katsumi; Yoshimura, Daisuke; Araki, Hiroshi; Kashida, Hiroshi; Hosokawa, Ayumu; Mori, Hirohito; Yamashita, Haruhiro; Motohashi, Osamu; Kobayashi, Kazuhiko; Hirayama, Michiaki; Kobayashi, Hiroyuki; Endo, Masaki; Yamano, Hiroo; Murakami, Kazunari; Koike, Tomoyuki; Hirasawa, Kingo; Miyaoka, Youichi; Hamamoto, Hidetaka; Hikichi, Takuto; Hanabata, Norihiro; Shimoda, Ryo; Hori, Shinichiro; Sato, Tadashi; Kodashima, Shinya; Okada, Hiroyuki; Mannami, Tomohiko; Yamamoto, Shojiro; Niwa, Yasumasa; Yashima, Kazuo; Tanabe, Satoshi; Satoh, Hiro; Sasaki, Fumisato; Yamazato, Tetsuro; Ikeda, Yoshiou; Nishisaki, Hogara; Nakagawa, Masahiro; Matsuda, Akio; Tamura, Fumio; Nishiyama, Hitoshi; Arita, Keiko; Kawasaki, Keisuke; Hoppo, Kazushige; Oka, Masashi; Ishihara, Shinichi; Mukasa, Michita; Minamino, Hiroaki; Yao, Kenshi

2017-10-01

Background and study aim  Magnifying narrow-band imaging (M-NBI) is useful for the accurate diagnosis of early gastric cancer (EGC). However, acquiring skill at M-NBI diagnosis takes substantial effort. An Internet-based e-learning system to teach endoscopic diagnosis of EGC using M-NBI has been developed. This study evaluated its effectiveness. Participants and methods  This study was designed as a multicenter randomized controlled trial. We recruited endoscopists as participants from all over Japan. After completing Test 1, which consisted of M-NBI images of 40 gastric lesions, participants were randomly assigned to the e-learning or non-e-learning groups. Only the e-learning group was allowed to access the e-learning system. After the e-learning period, both groups received Test 2. The analysis set was participants who scored < 80 % accuracy on Test 1. The primary end point was the difference in accuracy between Test 1 and Test 2 for the two groups. Results  A total of 395 participants from 77 institutions completed Test 1 (198 in the e-learning group and 197 in the non-e-learning group). After the e-learning period, all 395 completed Test 2. The analysis sets were e-learning group: n = 184; and non-e-learning group: n = 184. The mean Test 1 score was 59.9 % for the e-learning group and 61.7 % for the non-e-learning group. The change in accuracy in Test 2 was significantly higher in the e-learning group than in the non-e-learning group (7.4 points vs. 0.14 points, respectively; P  < 0.001). Conclusion This study clearly demonstrated the efficacy of the e-learning system in improving practitioners' capabilities to diagnose EGC using M-NBI.Trial registered at University Hospital Medical Information Network Clinical Trials Registry (UMIN000008569). © Georg Thieme Verlag KG Stuttgart · New York.

Self-Learning Intelligent Agents for Dynamic Traffic Routing on Transportation Networks

NASA Astrophysics Data System (ADS)

Sadek, Add; Basha, Nagi

Intelligent Transportation Systems (ITS) are designed to take advantage of recent advances in communications, electronics, and Information Technology in improving the efficiency and safety of transportation systems. Among the several ITS applications is the notion of Dynamic Traffic Routing (DTR), which involves generating "optimal" routing recommendations to drivers with the aim of maximizing network utilizing. In this paper, we demonstrate the feasibility of using a self-learning intelligent agent to solve the DTR problem to achieve traffic user equilibrium in a transportation network. The core idea is to deploy an agent to a simulation model of a highway. The agent then learns by itself by interacting with the simulation model. Once the agent reaches a satisfactory level of performance, it can then be deployed to the real-world, where it would continue to learn how to refine its control policies over time. To test this concept in this paper, the Cell Transmission Model (CTM) developed by Carlos Daganzo of the University of California at Berkeley is used to simulate a simple highway with two main alternative routes. With the model developed, a Reinforcement Learning Agent (RLA) is developed to learn how to best dynamically route traffic, so as to maximize the utilization of existing capacity. Preliminary results obtained from our experiments are promising. RL, being an adaptive online learning technique, appears to have a great potential for controlling a stochastic dynamic systems such as a transportation system. Furthermore, the approach is highly scalable and applicable to a variety of networks and roadways.

Implementation Challenges for Multivariable Control: What You Did Not Learn in School

NASA Technical Reports Server (NTRS)

Garg, Sanjay

2008-01-01

Multivariable control allows controller designs that can provide decoupled command tracking and robust performance in the presence of modeling uncertainties. Although the last two decades have seen extensive development of multivariable control theory and example applications to complex systems in software/hardware simulations, there are no production flying systems aircraft or spacecraft, that use multivariable control. This is because of the tremendous challenges associated with implementation of such multivariable control designs. Unfortunately, the curriculum in schools does not provide sufficient time to be able to provide an exposure to the students in such implementation challenges. The objective of this paper is to share the lessons learned by a practitioner of multivariable control in the process of applying some of the modern control theory to the Integrated Flight Propulsion Control (IFPC) design for an advanced Short Take-Off Vertical Landing (STOVL) aircraft simulation.

Self-teaching neural network learns difficult reactor control problem

DOE Office of Scientific and Technical Information (OSTI.GOV)

Jouse, W.C.

1989-01-01

A self-teaching neural network used as an adaptive controller quickly learns to control an unstable reactor configuration. The network models the behavior of a human operator. It is trained by allowing it to operate the reactivity control impulsively. It is punished whenever either the power or fuel temperature stray outside technical limits. Using a simple paradigm, the network constructs an internal representation of the punishment and of the reactor system. The reactor is constrained to small power orbits.

Learning for intelligent mobile robots

NASA Astrophysics Data System (ADS)

Hall, Ernest L.; Liao, Xiaoqun; Alhaj Ali, Souma M.

2003-10-01

Unlike intelligent industrial robots which often work in a structured factory setting, intelligent mobile robots must often operate in an unstructured environment cluttered with obstacles and with many possible action paths. However, such machines have many potential applications in medicine, defense, industry and even the home that make their study important. Sensors such as vision are needed. However, in many applications some form of learning is also required. The purpose of this paper is to present a discussion of recent technical advances in learning for intelligent mobile robots. During the past 20 years, the use of intelligent industrial robots that are equipped not only with motion control systems but also with sensors such as cameras, laser scanners, or tactile sensors that permit adaptation to a changing environment has increased dramatically. However, relatively little has been done concerning learning. Adaptive and robust control permits one to achieve point to point and controlled path operation in a changing environment. This problem can be solved with a learning control. In the unstructured environment, the terrain and consequently the load on the robot"s motors are constantly changing. Learning the parameters of a proportional, integral and derivative controller (PID) and artificial neural network provides an adaptive and robust control. Learning may also be used for path following. Simulations that include learning may be conducted to see if a robot can learn its way through a cluttered array of obstacles. If a situation is performed repetitively, then learning can also be used in the actual application. To reach an even higher degree of autonomous operation, a new level of learning is required. Recently learning theories such as the adaptive critic have been proposed. In this type of learning a critic provides a grade to the controller of an action module such as a robot. The creative control process is used that is "beyond the adaptive critic." A mathematical model of the creative control process is presented that illustrates the use for mobile robots. Examples from a variety of intelligent mobile robot applications are also presented. The significance of this work is in providing a greater understanding of the applications of learning to mobile robots that could lead to many applications.

[The mirror neuron system in motor and sensory rehabilitation].

PubMed

Oouchida, Yutaka; Izumi, Shinichi

2014-06-01

The discovery of the mirror neuron system has dramatically changed the study of motor control in neuroscience. The mirror neuron system provides a conceptual framework covering the aspects of motor as well as sensory functions in motor control. Previous studies of motor control can be classified as studies of motor or sensory functions, and these two classes of studies appear to have advanced independently. In rehabilitation requiring motor learning, such as relearning movement after limb paresis, however, sensory information of feedback for motor output as well as motor command are essential. During rehabilitation from chronic pain, motor exercise is one of the most effective treatments for pain caused by dysfunction in the sensory system. In rehabilitation where total intervention unifying the motor and sensory aspects of motor control is important, learning through imitation, which is associated with the mirror neuron system can be effective and suitable. In this paper, we introduce the clinical applications of imitated movement in rehabilitation from motor impairment after brain damage and phantom limb pain after limb amputation.

Iterative learning control with applications in energy generation, lasers and health care.

PubMed

Rogers, E; Tutty, O R

2016-09-01

Many physical systems make repeated executions of the same finite time duration task. One example is a robot in a factory or warehouse whose task is to collect an object in sequence from a location, transfer it over a finite duration, place it at a specified location or on a moving conveyor and then return for the next one and so on. Iterative learning control was especially developed for systems with this mode of operation and this paper gives an overview of this control design method using relatively recent relevant applications in wind turbines, free-electron lasers and health care, as exemplars to demonstrate its applicability.

Intelligent Control Systems Research

NASA Technical Reports Server (NTRS)

Loparo, Kenneth A.

1994-01-01

Results of a three phase research program into intelligent control systems are presented. The first phase looked at implementing the lowest or direct level of a hierarchical control scheme using a reinforcement learning approach assuming no a priori information about the system under control. The second phase involved the design of an adaptive/optimizing level of the hierarchy and its interaction with the direct control level. The third and final phase of the research was aimed at combining the results of the previous phases with some a priori information about the controlled system.

Working-memory capacity protects model-based learning from stress.

PubMed

Otto, A Ross; Raio, Candace M; Chiang, Alice; Phelps, Elizabeth A; Daw, Nathaniel D

2013-12-24

Accounts of decision-making have long posited the operation of separate, competing valuation systems in the control of choice behavior. Recent theoretical and experimental advances suggest that this classic distinction between habitual and goal-directed (or more generally, automatic and controlled) choice may arise from two computational strategies for reinforcement learning, called model-free and model-based learning. Popular neurocomputational accounts of reward processing emphasize the involvement of the dopaminergic system in model-free learning and prefrontal, central executive-dependent control systems in model-based choice. Here we hypothesized that the hypothalamic-pituitary-adrenal (HPA) axis stress response--believed to have detrimental effects on prefrontal cortex function--should selectively attenuate model-based contributions to behavior. To test this, we paired an acute stressor with a sequential decision-making task that affords distinguishing the relative contributions of the two learning strategies. We assessed baseline working-memory (WM) capacity and used salivary cortisol levels to measure HPA axis stress response. We found that stress response attenuates the contribution of model-based, but not model-free, contributions to behavior. Moreover, stress-induced behavioral changes were modulated by individual WM capacity, such that low-WM-capacity individuals were more susceptible to detrimental stress effects than high-WM-capacity individuals. These results enrich existing accounts of the interplay between acute stress, working memory, and prefrontal function and suggest that executive function may be protective against the deleterious effects of acute stress.

Working-memory capacity protects model-based learning from stress

PubMed Central

Otto, A. Ross; Raio, Candace M.; Chiang, Alice; Phelps, Elizabeth A.; Daw, Nathaniel D.

2013-01-01

Accounts of decision-making have long posited the operation of separate, competing valuation systems in the control of choice behavior. Recent theoretical and experimental advances suggest that this classic distinction between habitual and goal-directed (or more generally, automatic and controlled) choice may arise from two computational strategies for reinforcement learning, called model-free and model-based learning. Popular neurocomputational accounts of reward processing emphasize the involvement of the dopaminergic system in model-free learning and prefrontal, central executive–dependent control systems in model-based choice. Here we hypothesized that the hypothalamic-pituitary-adrenal (HPA) axis stress response—believed to have detrimental effects on prefrontal cortex function—should selectively attenuate model-based contributions to behavior. To test this, we paired an acute stressor with a sequential decision-making task that affords distinguishing the relative contributions of the two learning strategies. We assessed baseline working-memory (WM) capacity and used salivary cortisol levels to measure HPA axis stress response. We found that stress response attenuates the contribution of model-based, but not model-free, contributions to behavior. Moreover, stress-induced behavioral changes were modulated by individual WM capacity, such that low-WM-capacity individuals were more susceptible to detrimental stress effects than high-WM-capacity individuals. These results enrich existing accounts of the interplay between acute stress, working memory, and prefrontal function and suggest that executive function may be protective against the deleterious effects of acute stress. PMID:24324166

Reinforcement learning for adaptive optimal control of unknown continuous-time nonlinear systems with input constraints

NASA Astrophysics Data System (ADS)

Yang, Xiong; Liu, Derong; Wang, Ding

2014-03-01

In this paper, an adaptive reinforcement learning-based solution is developed for the infinite-horizon optimal control problem of constrained-input continuous-time nonlinear systems in the presence of nonlinearities with unknown structures. Two different types of neural networks (NNs) are employed to approximate the Hamilton-Jacobi-Bellman equation. That is, an recurrent NN is constructed to identify the unknown dynamical system, and two feedforward NNs are used as the actor and the critic to approximate the optimal control and the optimal cost, respectively. Based on this framework, the action NN and the critic NN are tuned simultaneously, without the requirement for the knowledge of system drift dynamics. Moreover, by using Lyapunov's direct method, the weights of the action NN and the critic NN are guaranteed to be uniformly ultimately bounded, while keeping the closed-loop system stable. To demonstrate the effectiveness of the present approach, simulation results are illustrated.

Effect of chunk strength on the performance of children with developmental dyslexia on artificial grammar learning task may be related to complexity.

PubMed

Schiff, Rachel; Katan, Pesia; Sasson, Ayelet; Kahta, Shani

2017-07-01

There's a long held view that chunks play a crucial role in artificial grammar learning performance. We compared chunk strength influences on performance, in high and low topological entropy (a measure of complexity) grammar systems, with dyslexic children, age-matched and reading-level-matched control participants. Findings show that age-matched control participants' performance reflected equivalent influence of chunk strength in the two topological entropy conditions, as typically found in artificial grammar learning experiments. By contrast, dyslexic children and reading-level-matched controls' performance reflected knowledge of chunk strength only under the low topological entropy condition. In the low topological entropy grammar system, they appeared completely unable to utilize chunk strength to make appropriate test item selections. In line with previous research, this study suggests that for typically developing children, it is the chunks that are attended during artificial grammar learning and create a foundation on which implicit associative learning mechanisms operate, and these chunks are unitized to different strengths. However, for children with dyslexia, it is complexity that may influence the subsequent memorability of chunks, independently of their strength.

Gemini Observatory base facility operations: systems engineering process and lessons learned

NASA Astrophysics Data System (ADS)

Serio, Andrew; Cordova, Martin; Arriagada, Gustavo; Adamson, Andy; Close, Madeline; Coulson, Dolores; Nitta, Atsuko; Nunez, Arturo

2016-08-01

Gemini North Observatory successfully began nighttime remote operations from the Hilo Base Facility control room in November 2015. The implementation of the Gemini North Base Facility Operations (BFO) products was a great learning experience for many of our employees, including the author of this paper, the BFO Systems Engineer. In this paper we focus on the tailored Systems Engineering processes used for the project, the various software tools used in project support, and finally discuss the lessons learned from the Gemini North implementation. This experience and the lessons learned will be used both to aid our implementation of the Gemini South BFO in 2016, and in future technical projects at Gemini Observatory.

A neural fuzzy controller learning by fuzzy error propagation

NASA Technical Reports Server (NTRS)

Nauck, Detlef; Kruse, Rudolf

1992-01-01

In this paper, we describe a procedure to integrate techniques for the adaptation of membership functions in a linguistic variable based fuzzy control environment by using neural network learning principles. This is an extension to our work. We solve this problem by defining a fuzzy error that is propagated back through the architecture of our fuzzy controller. According to this fuzzy error and the strength of its antecedent each fuzzy rule determines its amount of error. Depending on the current state of the controlled system and the control action derived from the conclusion, each rule tunes the membership functions of its antecedent and its conclusion. By this we get an unsupervised learning technique that enables a fuzzy controller to adapt to a control task by knowing just about the global state and the fuzzy error.

Discrete Serotonin Systems Mediate Memory Enhancement and Escape Latencies after Unpredicted Aversive Experience in Drosophila Place Memory

PubMed Central

Sitaraman, Divya; Kramer, Elizabeth F.; Kahsai, Lily; Ostrowski, Daniela; Zars, Troy

2017-01-01

Feedback mechanisms in operant learning are critical for animals to increase reward or reduce punishment. However, not all conditions have a behavior that can readily resolve an event. Animals must then try out different behaviors to better their situation through outcome learning. This form of learning allows for novel solutions and with positive experience can lead to unexpected behavioral routines. Learned helplessness, as a type of outcome learning, manifests in part as increases in escape latency in the face of repeated unpredicted shocks. Little is known about the mechanisms of outcome learning. When fruit fly Drosophila melanogaster are exposed to unpredicted high temperatures in a place learning paradigm, flies both increase escape latencies and have a higher memory when given control of a place/temperature contingency. Here we describe discrete serotonin neuronal circuits that mediate aversive reinforcement, escape latencies, and memory levels after place learning in the presence and absence of unexpected aversive events. The results show that two features of learned helplessness depend on the same modulatory system as aversive reinforcement. Moreover, changes in aversive reinforcement and escape latency depend on local neural circuit modulation, while memory enhancement requires larger modulation of multiple behavioral control circuits. PMID:29321732

Intelligent flight control systems

NASA Technical Reports Server (NTRS)

Stengel, Robert F.

1993-01-01

The capabilities of flight control systems can be enhanced by designing them to emulate functions of natural intelligence. Intelligent control functions fall in three categories. Declarative actions involve decision-making, providing models for system monitoring, goal planning, and system/scenario identification. Procedural actions concern skilled behavior and have parallels in guidance, navigation, and adaptation. Reflexive actions are spontaneous, inner-loop responses for control and estimation. Intelligent flight control systems learn knowledge of the aircraft and its mission and adapt to changes in the flight environment. Cognitive models form an efficient basis for integrating 'outer-loop/inner-loop' control functions and for developing robust parallel-processing algorithms.

Using a collaborative Mobile Augmented Reality learning application (CoMARLA) to improve Improve Student Learning

NASA Astrophysics Data System (ADS)

Hanafi, Hafizul Fahri bin; Soh Said, Che; Hanee Ariffin, Asma; Azlan Zainuddin, Nur; Samsuddin, Khairulanuar

2016-11-01

This study was carried out to improve student learning in ICT course using a collaborative mobile augmented reality learning application (CoMARLA). This learning application was developed based on the constructivist framework that would engender collaborative learning environment, in which students could learn collaboratively using their mobile phones. The research design was based on the pretest posttest control group design. The dependent variable was students’ learning performance after learning, and the independent variables were learning method and gender. Students’ learning performance before learning was treated as the covariate. The sample of the study comprised 120 non-IT (non-technical) undergraduates, with the mean age of 19.5. They were randomized into two groups, namely the experimental and control group. The experimental group used CoMARLA to learn one of the topics of the ICT Literacy course, namely Computer System; whereas the control group learned using the conventional approach. The research instrument used was a set of multiple-choice questions pertaining to the above topic. Pretesting was carried out before the learning sessions, and posttesting was performed after 6 hours of learning. Using the SPSS, Analysis of Covariance (ANCOVA) was performed on the data. The analysis showed that there were main effects attributed to the learning method and gender. The experimental group outperformed the control group by almost 9%, and male students outstripped their opposite counterparts by as much as 3%. Furthermore, an interaction effect was also observed showing differential performances of male students based on the learning methods, which did not occur among female students. Hence, the tool can be used to help undergraduates learn with greater efficacy when contextualized in an appropriate setting.

A vehicle stability control strategy with adaptive neural network sliding mode theory based on system uncertainty approximation

NASA Astrophysics Data System (ADS)

Ji, Xuewu; He, Xiangkun; Lv, Chen; Liu, Yahui; Wu, Jian

2018-06-01

Modelling uncertainty, parameter variation and unknown external disturbance are the major concerns in the development of an advanced controller for vehicle stability at the limits of handling. Sliding mode control (SMC) method has proved to be robust against parameter variation and unknown external disturbance with satisfactory tracking performance. But modelling uncertainty, such as errors caused in model simplification, is inevitable in model-based controller design, resulting in lowered control quality. The adaptive radial basis function network (ARBFN) can effectively improve the control performance against large system uncertainty by learning to approximate arbitrary nonlinear functions and ensure the global asymptotic stability of the closed-loop system. In this paper, a novel vehicle dynamics stability control strategy is proposed using the adaptive radial basis function network sliding mode control (ARBFN-SMC) to learn system uncertainty and eliminate its adverse effects. This strategy adopts a hierarchical control structure which consists of reference model layer, yaw moment control layer, braking torque allocation layer and executive layer. Co-simulation using MATLAB/Simulink and AMESim is conducted on a verified 15-DOF nonlinear vehicle system model with the integrated-electro-hydraulic brake system (I-EHB) actuator in a Sine With Dwell manoeuvre. The simulation results show that ARBFN-SMC scheme exhibits superior stability and tracking performance in different running conditions compared with SMC scheme.

Psychological and Organizational Variables Associated with Workplace Learning in Small and Medium Manufacturing Businesses in Korea

ERIC Educational Resources Information Center

Moon, Se-Yeon; Na, Seung-Il

2009-01-01

The purpose of this study was to determine the relationship between workplace learning and psychological variables, such as learning competency, motivation, curiosity, self-esteem and locus of control, and organizational variables, such as centralization of power, formality, merit system and communication. The studied population consisted entirely…

Peer Assisted Learning in the Clinical Setting: An Activity Systems Analysis

ERIC Educational Resources Information Center

Bennett, Deirdre; O'Flynn, Siun; Kelly, Martina

2015-01-01

Peer assisted learning (PAL) is a common feature of medical education. Understanding of PAL has been based on processes and outcomes in controlled settings, such as clinical skills labs. PAL in the clinical setting, a complex learning environment, requires fresh evaluation. Socio-cultural theory is proposed as a means to understand educational…

Learning about Computer-Based Education in Adult Basic Education.

ERIC Educational Resources Information Center

Fahy, Patrick J.

In 1979 the adult basic education department at the Alberta Vocational Centre (AVC), Edmonton, began to use the Control Data PLATO system. Results of the first PLATO project showed students using PLATO learned at least as much as students in regular classes. Students learned faster and reported great satisfaction with PLATO experiences. Staff and…

Supporting Interaction among Participants of Online Learning Using the Knowledge Sharing Concept

ERIC Educational Resources Information Center

Chao, Chih-Yang; Hwu, Shiow-Lin; Chang, Chi-Cheng

2011-01-01

In education business, proper interaction is a crucial factor for learning effectiveness. However, it is difficult to successfully guide the participants to achieve the appropriate interaction in an online learning environment. That is, the interaction as well as internal dialogue should be systemically performed under a valid control. In this…

The Reading Disc: Learning to Read Using Interactive CD.

ERIC Educational Resources Information Center

Shaw, Simon

1991-01-01

Describes the development of an interactive compact disc on CD-ROM XA that was designed to help adults learn to read. The application of technology to learning is discussed, differences in learner control in computer-based systems are considered, virtual writing is described, and assessment activities available on the disc are explained. (five…

Simon Fraser University's New Interactive Learning System to Teach French as a Second Language.

ERIC Educational Resources Information Center

Kirchner, Glenn

1988-01-01

Provides an overview of the design, production, and preliminary testing of a microcomputer-controlled interactive learning workstation developed at Simon Fraser University to teach French as a Second Language. Criteria and guidelines are discussed; the authoring system is explained; and field testing with grades four through seven is described.…

Fault-tolerant optimised tracking control for unknown discrete-time linear systems using a combined reinforcement learning and residual compensation methodology

NASA Astrophysics Data System (ADS)

Han, Ke-Zhen; Feng, Jian; Cui, Xiaohong

2017-10-01

This paper considers the fault-tolerant optimised tracking control (FTOTC) problem for unknown discrete-time linear system. A research scheme is proposed on the basis of data-based parity space identification, reinforcement learning and residual compensation techniques. The main characteristic of this research scheme lies in the parity-space-identification-based simultaneous tracking control and residual compensation. The specific technical line consists of four main contents: apply subspace aided method to design observer-based residual generator; use reinforcement Q-learning approach to solve optimised tracking control policy; rely on robust H∞ theory to achieve noise attenuation; adopt fault estimation triggered by residual generator to perform fault compensation. To clarify the design and implementation procedures, an integrated algorithm is further constructed to link up these four functional units. The detailed analysis and proof are subsequently given to explain the guaranteed FTOTC performance of the proposed conclusions. Finally, a case simulation is provided to verify its effectiveness.

Ego depletion interferes with rule-defined category learning but not non-rule-defined category learning.

PubMed

Minda, John P; Rabi, Rahel

2015-01-01

Considerable research on category learning has suggested that many cognitive and environmental factors can have a differential effect on the learning of rule-defined (RD) categories as opposed to the learning of non-rule-defined (NRD) categories. Prior research has also suggested that ego depletion can temporarily reduce the capacity for executive functioning and cognitive flexibility. The present study examined whether temporarily reducing participants' executive functioning via a resource depletion manipulation would differentially impact RD and NRD category learning. Participants were either asked to write a story with no restrictions (the control condition), or without using two common letters (the ego depletion condition). Participants were then asked to learn either a set of RD categories or a set of NRD categories. Resource depleted participants performed more poorly than controls on the RD task, but did not differ from controls on the NRD task, suggesting that self regulatory resources are required for successful RD category learning. These results lend support to multiple systems theories and clarify the role of self-regulatory resources within this theory.

Ego depletion interferes with rule-defined category learning but not non-rule-defined category learning

PubMed Central

Minda, John P.; Rabi, Rahel

2015-01-01

Considerable research on category learning has suggested that many cognitive and environmental factors can have a differential effect on the learning of rule-defined (RD) categories as opposed to the learning of non-rule-defined (NRD) categories. Prior research has also suggested that ego depletion can temporarily reduce the capacity for executive functioning and cognitive flexibility. The present study examined whether temporarily reducing participants’ executive functioning via a resource depletion manipulation would differentially impact RD and NRD category learning. Participants were either asked to write a story with no restrictions (the control condition), or without using two common letters (the ego depletion condition). Participants were then asked to learn either a set of RD categories or a set of NRD categories. Resource depleted participants performed more poorly than controls on the RD task, but did not differ from controls on the NRD task, suggesting that self regulatory resources are required for successful RD category learning. These results lend support to multiple systems theories and clarify the role of self-regulatory resources within this theory. PMID:25688220

A reductionist approach to the analysis of learning in brain-computer interfaces.

PubMed

Danziger, Zachary

2014-04-01

The complexity and scale of brain-computer interface (BCI) studies limit our ability to investigate how humans learn to use BCI systems. It also limits our capacity to develop adaptive algorithms needed to assist users with their control. Adaptive algorithm development is forced offline and typically uses static data sets. But this is a poor substitute for the online, dynamic environment where algorithms are ultimately deployed and interact with an adapting user. This work evaluates a paradigm that simulates the control problem faced by human subjects when controlling a BCI, but which avoids the many complications associated with full-scale BCI studies. Biological learners can be studied in a reductionist way as they solve BCI-like control problems, and machine learning algorithms can be developed and tested in closed loop with the subjects before being translated to full BCIs. The method is to map 19 joint angles of the hand (representing neural signals) to the position of a 2D cursor which must be piloted to displayed targets (a typical BCI task). An investigation is presented on how closely the joint angle method emulates BCI systems; a novel learning algorithm is evaluated, and a performance difference between genders is discussed.

Synchronization of Chaotic Systems without Direct Connections Using Reinforcement Learning

NASA Astrophysics Data System (ADS)

Sato, Norihisa; Adachi, Masaharu

In this paper, we propose a control method for the synchronization of chaotic systems that does not require the systems to be connected, unlike existing methods such as that proposed by Pecora and Carroll in 1990. The method is based on the reinforcement learning algorithm. We apply our method to two discrete-time chaotic systems with mismatched parameters and achieve M step delay synchronization. Moreover, we extend the proposed method to the synchronization of continuous-time chaotic systems.

On Mixed Data and Event Driven Design for Adaptive-Critic-Based Nonlinear $H_{\\infty}$ Control.

PubMed

Wang, Ding; Mu, Chaoxu; Liu, Derong; Ma, Hongwen

2018-04-01

In this paper, based on the adaptive critic learning technique, the control for a class of unknown nonlinear dynamic systems is investigated by adopting a mixed data and event driven design approach. The nonlinear control problem is formulated as a two-player zero-sum differential game and the adaptive critic method is employed to cope with the data-based optimization. The novelty lies in that the data driven learning identifier is combined with the event driven design formulation, in order to develop the adaptive critic controller, thereby accomplishing the nonlinear control. The event driven optimal control law and the time driven worst case disturbance law are approximated by constructing and tuning a critic neural network. Applying the event driven feedback control, the closed-loop system is built with stability analysis. Simulation studies are conducted to verify the theoretical results and illustrate the control performance. It is significant to observe that the present research provides a new avenue of integrating data-based control and event-triggering mechanism into establishing advanced adaptive critic systems.

Modelling, Simulation, Animation, and Real-Time Control (Mosart) for a Class of Electromechanical Systems: A System-Theoretic Approach

ERIC Educational Resources Information Center

Rodriguez, Armando A.; Metzger, Richard P.; Cifdaloz, Oguzhan; Dhirasakdanon, Thanate; Welfert, Bruno

2004-01-01

This paper describes an interactive modelling, simulation, animation, and real-time control (MoSART) environment for a class of 'cart-pendulum' electromechanical systems that may be used to enhance learning within differential equations and linear algebra classes. The environment is useful for conveying fundamental mathematical/systems concepts…

Effective Information Extraction Framework for Heterogeneous Clinical Reports Using Online Machine Learning and Controlled Vocabularies

PubMed Central

Zheng, Shuai; Ghasemzadeh, Nima; Hayek, Salim S; Quyyumi, Arshed A

2017-01-01

Background Extracting structured data from narrated medical reports is challenged by the complexity of heterogeneous structures and vocabularies and often requires significant manual effort. Traditional machine-based approaches lack the capability to take user feedbacks for improving the extraction algorithm in real time. Objective Our goal was to provide a generic information extraction framework that can support diverse clinical reports and enables a dynamic interaction between a human and a machine that produces highly accurate results. Methods A clinical information extraction system IDEAL-X has been built on top of online machine learning. It processes one document at a time, and user interactions are recorded as feedbacks to update the learning model in real time. The updated model is used to predict values for extraction in subsequent documents. Once prediction accuracy reaches a user-acceptable threshold, the remaining documents may be batch processed. A customizable controlled vocabulary may be used to support extraction. Results Three datasets were used for experiments based on report styles: 100 cardiac catheterization procedure reports, 100 coronary angiographic reports, and 100 integrated reports—each combines history and physical report, discharge summary, outpatient clinic notes, outpatient clinic letter, and inpatient discharge medication report. Data extraction was performed by 3 methods: online machine learning, controlled vocabularies, and a combination of these. The system delivers results with F1 scores greater than 95%. Conclusions IDEAL-X adopts a unique online machine learning–based approach combined with controlled vocabularies to support data extraction for clinical reports. The system can quickly learn and improve, thus it is highly adaptable. PMID:28487265

Sample Manipulation System for Sample Analysis at Mars

NASA Technical Reports Server (NTRS)

Mumm, Erik; Kennedy, Tom; Carlson, Lee; Roberts, Dustyn

2008-01-01

The Sample Analysis at Mars (SAM) instrument will analyze Martian samples collected by the Mars Science Laboratory Rover with a suite of spectrometers. This paper discusses the driving requirements, design, and lessons learned in the development of the Sample Manipulation System (SMS) within SAM. The SMS stores and manipulates 74 sample cups to be used for solid sample pyrolysis experiments. Focus is given to the unique mechanism architecture developed to deliver a high packing density of sample cups in a reliable, fault tolerant manner while minimizing system mass and control complexity. Lessons learned are presented on contamination control, launch restraint mechanisms for fragile sample cups, and mechanism test data.

Genetic algorithms for adaptive real-time control in space systems

NASA Technical Reports Server (NTRS)

Vanderzijp, J.; Choudry, A.

1988-01-01

Genetic Algorithms that are used for learning as one way to control the combinational explosion associated with the generation of new rules are discussed. The Genetic Algorithm approach tends to work best when it can be applied to a domain independent knowledge representation. Applications to real time control in space systems are discussed.

Learning Multirobot Hose Transportation and Deployment by Distributed Round-Robin Q-Learning.

PubMed

Fernandez-Gauna, Borja; Etxeberria-Agiriano, Ismael; Graña, Manuel

2015-01-01

Multi-Agent Reinforcement Learning (MARL) algorithms face two main difficulties: the curse of dimensionality, and environment non-stationarity due to the independent learning processes carried out by the agents concurrently. In this paper we formalize and prove the convergence of a Distributed Round Robin Q-learning (D-RR-QL) algorithm for cooperative systems. The computational complexity of this algorithm increases linearly with the number of agents. Moreover, it eliminates environment non sta tionarity by carrying a round-robin scheduling of the action selection and execution. That this learning scheme allows the implementation of Modular State-Action Vetoes (MSAV) in cooperative multi-agent systems, which speeds up learning convergence in over-constrained systems by vetoing state-action pairs which lead to undesired termination states (UTS) in the relevant state-action subspace. Each agent's local state-action value function learning is an independent process, including the MSAV policies. Coordination of locally optimal policies to obtain the global optimal joint policy is achieved by a greedy selection procedure using message passing. We show that D-RR-QL improves over state-of-the-art approaches, such as Distributed Q-Learning, Team Q-Learning and Coordinated Reinforcement Learning in a paradigmatic Linked Multi-Component Robotic System (L-MCRS) control problem: the hose transportation task. L-MCRS are over-constrained systems with many UTS induced by the interaction of the passive linking element and the active mobile robots.

Fuzzy Q-Learning for Generalization of Reinforcement Learning

NASA Technical Reports Server (NTRS)

Berenji, Hamid R.

1996-01-01

Fuzzy Q-Learning, introduced earlier by the author, is an extension of Q-Learning into fuzzy environments. GARIC is a methodology for fuzzy reinforcement learning. In this paper, we introduce GARIC-Q, a new method for doing incremental Dynamic Programming using a society of intelligent agents which are controlled at the top level by Fuzzy Q-Learning and at the local level, each agent learns and operates based on GARIC. GARIC-Q improves the speed and applicability of Fuzzy Q-Learning through generalization of input space by using fuzzy rules and bridges the gap between Q-Learning and rule based intelligent systems.

NASA Ares I Launch Vehicle Roll and Reaction Control Systems Lessons Learned

NASA Technical Reports Server (NTRS)

Butt, Adam; Popp, Chris G.; Jernigan, Frankie R.; Paseur, Lila F.; Pitts, Hank M.

2011-01-01

On April 15, 2010 President Barak Obama made the official announcement that the Constellation Program, which included the Ares I launch vehicle, would be canceled. NASA s Ares I launch vehicle was being designed to launch the Orion Crew Exploration Vehicle, returning humans to the moon, Mars, and beyond. It consisted of a First Stage (FS) five segment solid rocket booster and a liquid J-2X Upper Stage (US) engine. Roll control for the FS was planned to be handled by a dedicated Roll Control System (RoCS), located on the connecting interstage. Induced yaw or pitch moments experienced during FS ascent would have been handled by vectoring of the booster nozzle. After FS booster separation, the US Reaction Control System (ReCS) would have provided the US Element with three degrees of freedom control as needed. The lessons learned documented in this paper will be focused on the technical designs and producibility of both systems along with the partnership between NASA and Boeing, who was on contract to build the Ares I US Element, which included the FS RoCS and US ReCS. In regards to partnership, focus will be placed on integration along with technical work accomplished by Boeing with special emphasis on each task order. In summary, this paper attempts to capture key lessons learned that should be helpful in the development of future launch vehicle RCS designs.

Supervised learning from human performance at the computationally hard problem of optimal traffic signal control on a network of junctions

PubMed Central

Box, Simon

2014-01-01

Optimal switching of traffic lights on a network of junctions is a computationally intractable problem. In this research, road traffic networks containing signallized junctions are simulated. A computer game interface is used to enable a human ‘player’ to control the traffic light settings on the junctions within the simulation. A supervised learning approach, based on simple neural network classifiers can be used to capture human player's strategies in the game and thus develop a human-trained machine control (HuTMaC) system that approaches human levels of performance. Experiments conducted within the simulation compare the performance of HuTMaC to two well-established traffic-responsive control systems that are widely deployed in the developed world and also to a temporal difference learning-based control method. In all experiments, HuTMaC outperforms the other control methods in terms of average delay and variance over delay. The conclusion is that these results add weight to the suggestion that HuTMaC may be a viable alternative, or supplemental method, to approximate optimization for some practical engineering control problems where the optimal strategy is computationally intractable. PMID:26064570

Supervised learning from human performance at the computationally hard problem of optimal traffic signal control on a network of junctions.

PubMed

Box, Simon

2014-12-01

Optimal switching of traffic lights on a network of junctions is a computationally intractable problem. In this research, road traffic networks containing signallized junctions are simulated. A computer game interface is used to enable a human 'player' to control the traffic light settings on the junctions within the simulation. A supervised learning approach, based on simple neural network classifiers can be used to capture human player's strategies in the game and thus develop a human-trained machine control (HuTMaC) system that approaches human levels of performance. Experiments conducted within the simulation compare the performance of HuTMaC to two well-established traffic-responsive control systems that are widely deployed in the developed world and also to a temporal difference learning-based control method. In all experiments, HuTMaC outperforms the other control methods in terms of average delay and variance over delay. The conclusion is that these results add weight to the suggestion that HuTMaC may be a viable alternative, or supplemental method, to approximate optimization for some practical engineering control problems where the optimal strategy is computationally intractable.

Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting.

PubMed

Coop, Robert; Mishtal, Aaron; Arel, Itamar

2013-10-01

Catastrophic forgetting is a well-studied attribute of most parameterized supervised learning systems. A variation of this phenomenon, in the context of feedforward neural networks, arises when nonstationary inputs lead to loss of previously learned mappings. The majority of the schemes proposed in the literature for mitigating catastrophic forgetting were not data driven and did not scale well. We introduce the fixed expansion layer (FEL) feedforward neural network, which embeds a sparsely encoding hidden layer to help mitigate forgetting of prior learned representations. In addition, we investigate a novel framework for training ensembles of FEL networks, based on exploiting an information-theoretic measure of diversity between FEL learners, to further control undesired plasticity. The proposed methodology is demonstrated on a basic classification task, clearly emphasizing its advantages over existing techniques. The architecture proposed can be enhanced to address a range of computational intelligence tasks, such as regression problems and system control.

Action-based language: a theory of language acquisition, comprehension, and production.

PubMed

Glenberg, Arthur M; Gallese, Vittorio

2012-07-01

Evolution and the brain have done a marvelous job solving many tricky problems in action control, including problems of learning, hierarchical control over serial behavior, continuous recalibration, and fluency in the face of slow feedback. Given that evolution tends to be conservative, it should not be surprising that these solutions are exploited to solve other tricky problems, such as the design of a communication system. We propose that a mechanism of motor control, paired controller/predictor models, has been exploited for language learning, comprehension, and production. Our account addresses the development of grammatical regularities and perspective, as well as how linguistic symbols become meaningful through grounding in perception, action, and emotional systems. Copyright © 2011 Elsevier Srl. All rights reserved.

Remotely controlling of mobile robots using gesture captured by the Kinect and recognized by machine learning method

NASA Astrophysics Data System (ADS)

Hsu, Roy CHaoming; Jian, Jhih-Wei; Lin, Chih-Chuan; Lai, Chien-Hung; Liu, Cheng-Ting

2013-01-01

The main purpose of this paper is to use machine learning method and Kinect and its body sensation technology to design a simple, convenient, yet effective robot remote control system. In this study, a Kinect sensor is used to capture the human body skeleton with depth information, and a gesture training and identification method is designed using the back propagation neural network to remotely command a mobile robot for certain actions via the Bluetooth. The experimental results show that the designed mobile robots remote control system can achieve, on an average, more than 96% of accurate identification of 7 types of gestures and can effectively control a real e-puck robot for the designed commands.

Biologically Inspired SNN for Robot Control.

PubMed

Nichols, Eric; McDaid, Liam J; Siddique, Nazmul

2013-02-01

This paper proposes a spiking-neural-network-based robot controller inspired by the control structures of biological systems. Information is routed through the network using facilitating dynamic synapses with short-term plasticity. Learning occurs through long-term synaptic plasticity which is implemented using the temporal difference learning rule to enable the robot to learn to associate the correct movement with the appropriate input conditions. The network self-organizes to provide memories of environments that the robot encounters. A Pioneer robot simulator with laser and sonar proximity sensors is used to verify the performance of the network with a wall-following task, and the results are presented.

Neuromorphic learning of continuous-valued mappings from noise-corrupted data

NASA Technical Reports Server (NTRS)

Troudet, T.; Merrill, W.

1991-01-01

The effect of noise on the learning performance of the backpropagation algorithm is analyzed. A selective sampling of the training set is proposed to maximize the learning of control laws by backpropagation, when the data have been corrupted by noise. The training scheme is applied to the nonlinear control of a cart-pole system in the presence of noise. The neural computation provides the neurocontroller with good noise-filtering properties. In the presence of plant noise, the neurocontroller is found to be more stable than the teacher. A novel perspective on the application of neural network technology to control engineering is presented.

The Impact of an Assurance System on the Quality of Teaching and Learning--Using the Example of a University in Russia and One of the Universities in Germany

ERIC Educational Resources Information Center

Szymenderski, Peggy; Yagudina, Liliya; Burenkova, Olga

2015-01-01

In this paper we consider the question of how quality assurance can have a real, positive impact on the quality of teaching and learning at universities, considering the realities of different systems--the system of control and the system of quality culture--in using the example of two universities: the KNITU-KAI in Russia and the TU Dresden in…

A Survey of Research in Supervisory Control and Data Acquisition (SCADA)

DTIC Science & Technology

2014-09-01

distance learning .2 The data acquired may be operationally oriented and used to better run the system, or it could be strategic in nature and used to...Technically the SCADA system is composed of the information technology (IT) that provides the human- machine interface (HMI) and stores and analyzes the data...systems work by learning what normal or benign traffic is and reporting on any abnormal traffic. These systems have the potential to detect zero-day

Distributed Learning and Information Dynamics In Networked Autonomous Systems

DTIC Science & Technology

2015-11-20

2009 to June 30, 2015 4. TITLE AND SUBTITLE DISTRIBUTED LEARNING AND INFORMATION DYNAMICS IN NETWORKED AUTONOMOUS SYSTEMS 5a. CONTRACT NUMBER 5b...AUTONOMOUS SYSTEMS AFOSR Grant #FA9550–09–1–0538 PI: Eric Feron (current) Jeff S. Shamma (former) Georgia Institute of Technology Atlanta, GA 30332 1...Control. Design of event-based optimal remote estimation systems : We have proposed two new for- mulations to study the design of optimal remote

Lessons learned in control center technologies and non-technologies

NASA Technical Reports Server (NTRS)

Hansen, Elaine R.

1991-01-01

Information is given in viewgraph form on the Solar Mesosphere Explorer (SME) Control Center and the Oculometer and Automated Space Interface System (OASIS). Topics covered include SME mission operations functions; technical and non-technical features of the SME control center; general tasks and objects within the Space Station Freedom (SSF) ground system nodes; OASIS-Real Time for the control and monitoring of of space systems and subsystems; and OASIS planning, scheduling, and PC architecture.

Development of Servo Motor Trainer for Basic Control System in Laboratory of Electrical Engineering Control System Faculty of Engineering Universitas Negeri Surabaya

NASA Astrophysics Data System (ADS)

Endryansyah; Wanarti Rusimamto, Puput; Ridianto, Adam; Sugiarto, Hariyadi

2018-04-01

In the Department of Electrical Engineering FT Unesa, there are 3 majors: S1 Electrical Engineering Education, S1 Electrical Engineering, and D3 Electrical Engineering. Courses the Basic System Settings go to in the curriculum of the three programs. Team lecturer college of basic system settings seek learning innovation, focused on the development of trainer to student practicum at the laboratory of systems control. Trainer developed is a servo motor along with the lab module that contains a wide variety of theories about the servo motor and guide the practicum. This research type is development research using methods Research & development (R & D). In which the steps are applied in this study is as follows: pay attention to the potential and existing problems, gather information and study the literature, design the product, validate the design, revise the design, a limited trial. The results of the validation of learning device in the form of modules and trainer obtained as follows: score validation of learning device is 3,64; score validation lab module Servo Motor is 3,47; and questionnaire responses of students is 3,73. The result of the whole validation value is located in the interval >of 3.25 s/d 4 with the category of “Very Valid”, so it can be concluded that all instruments have a level of validity “Very Valid” and worthy of use for further learning.

The roles of the olivocerebellar pathway in motor learning and motor control. A consensus paper

PubMed Central

Lang, Eric J.; Apps, Richard; Bengtsson, Fredrik; Cerminara, Nadia L.; De Zeeuw, Chris I.; Ebner, Timothy J.; Heck, Detlef H.; Jaeger, Dieter; Jörntell, Henrik; Kawato, Mitsuo; Otis, Thomas S.; Ozyildirim, Ozgecan; Popa, Laurentiu S.; Reeves, Alexander M.B.; Schweighofer, Nicolas; Sugihara, Izumi; Xiao, Jianqiang

2016-01-01

For many decades the predominant view in the cerebellar field has been that the olivocerebellar system's primary function is to induce plasticity in the cerebellar cortex, specifically, at the parallel fiber-Purkinje cell synapse. However, it has also long been proposed that the olivocerebellar system participates directly in motor control by helping to shape ongoing motor commands being issued by the cerebellum. Evidence consistent with both hypotheses exists; however, they are often investigated as mutually exclusive alternatives. In contrast, here we take the perspective that the olivocerebellar system can contribute to both the motor learning and motor control functions of the cerebellum, and might also play a role in development. We then consider the potential problems and benefits of its having multiple functions. Moreover, we discuss how its distinctive characteristics (e.g., low firing rates, synchronization, variable complex spike waveform) make it more or less suitable for one or the other of these functions, and why its having a dual role makes sense from an evolutionary perspective. We did not attempt to reach a consensus on the specific role(s) the olivocerebellar system plays in different types of movements, as that will ultimately be determined experimentally; however, collectively, the various contributions highlight the flexibility of the olivocerebellar system, and thereby suggest it has the potential to act in both the motor learning and motor control functions of the cerebellum. PMID:27193702

Lessons Learned from the Advanced Topographic Laser Altimeter System

NASA Technical Reports Server (NTRS)

Garrison, Matt; Patel, Deepak; Bradshaw, Heather; Robinson, Frank; Neuberger, Dave

2016-01-01

The ICESat-2 Advanced Topographic Laser Altimeter System (ATLAS) instrument is an upcoming Earth Science mission focusing on the effects of climate change. The flight instrument passed all environmental testing at GSFC (Goddard Space Flight Center) and is now ready to be shipped to the spacecraft vendor for integration and testing. This presentation walks through the lessons learned from design, hardware, analysis and testing perspective. ATLAS lessons learned include general thermal design, analysis, hardware, and testing issues as well as lessons specific to laser systems, two-phase thermal control, and optical assemblies with precision alignment requirements.

An adaptive learning control system for large flexible structures

NASA Technical Reports Server (NTRS)

Thau, F. E.

1985-01-01

The objective of the research has been to study the design of adaptive/learning control systems for the control of large flexible structures. In the first activity an adaptive/learning control methodology for flexible space structures was investigated. The approach was based on using a modal model of the flexible structure dynamics and an output-error identification scheme to identify modal parameters. In the second activity, a least-squares identification scheme was proposed for estimating both modal parameters and modal-to-actuator and modal-to-sensor shape functions. The technique was applied to experimental data obtained from the NASA Langley beam experiment. In the third activity, a separable nonlinear least-squares approach was developed for estimating the number of excited modes, shape functions, modal parameters, and modal amplitude and velocity time functions for a flexible structure. In the final research activity, a dual-adaptive control strategy was developed for regulating the modal dynamics and identifying modal parameters of a flexible structure. A min-max approach was used for finding an input to provide modal parameter identification while not exceeding reasonable bounds on modal displacement.

Learning effects of dynamic postural control by auditory biofeedback versus visual biofeedback training.

PubMed

Hasegawa, Naoya; Takeda, Kenta; Sakuma, Moe; Mani, Hiroki; Maejima, Hiroshi; Asaka, Tadayoshi

2017-10-01

Augmented sensory biofeedback (BF) for postural control is widely used to improve postural stability. However, the effective sensory information in BF systems of motor learning for postural control is still unknown. The purpose of this study was to investigate the learning effects of visual versus auditory BF training in dynamic postural control. Eighteen healthy young adults were randomly divided into two groups (visual BF and auditory BF). In test sessions, participants were asked to bring the real-time center of pressure (COP) in line with a hidden target by body sway in the sagittal plane. The target moved in seven cycles of sine curves at 0.23Hz in the vertical direction on a monitor. In training sessions, the visual and auditory BF groups were required to change the magnitude of a visual circle and a sound, respectively, according to the distance between the COP and target in order to reach the target. The perceptual magnitudes of visual and auditory BF were equalized according to Stevens' power law. At the retention test, the auditory but not visual BF group demonstrated decreased postural performance errors in both the spatial and temporal parameters under the no-feedback condition. These findings suggest that visual BF increases the dependence on visual information to control postural performance, while auditory BF may enhance the integration of the proprioceptive sensory system, which contributes to motor learning without BF. These results suggest that auditory BF training improves motor learning of dynamic postural control. Copyright © 2017 Elsevier B.V. All rights reserved.

Learning from the spinal cord: How the study of spinal cord plasticity informs our view of learning

PubMed Central

Grau, James W.

2013-01-01

The paper reviews research examining whether and how training can induce a lasting change in spinal cord function. A framework for the study of learning, and some essential issues in experimental design, are discussed. A core element involves delayed assessment under common conditions. Research has shown that brain systems can induce a lasting (memory-like) alteration in spinal function. Neurons within the lower (lumbosacral) spinal cord can also adapt when isolated from the brain by means of a thoracic transection. Using traditional learning paradigms, evidence suggests that spinal neurons support habituation and sensitization as well as Pavlovian and instrumental conditioning. At a neurobiological level, spinal systems support phenomena (e.g., long-term potentiation), and involve mechanisms (e.g., NMDA mediated plasticity, protein synthesis) implicated in brain-dependent learning and memory. Spinal learning also induces modulatory effects that alter the capacity for learning. Uncontrollable/unpredictable stimulation disables the capacity for instrumental learning and this effect has been linked to the cytokine tumor necrosis factor (TNF). Predictable/controllable stimulation enables learning and counters the adverse effects of uncontrollable simulation through a process that depends upon brain-derived neurotrophic factor (BDNF). Finally, uncontrollable, but not controllable, nociceptive stimulation impairs recovery after a contusion injury. A process-oriented approach (neurofunctionalism) is outlined that encourages a broader view of learning phenomena. PMID:23973905

HyFIS: adaptive neuro-fuzzy inference systems and their application to nonlinear dynamical systems.

PubMed

Kim, J; Kasabov, N

1999-11-01

This paper proposes an adaptive neuro-fuzzy system, HyFIS (Hybrid neural Fuzzy Inference System), for building and optimising fuzzy models. The proposed model introduces the learning power of neural networks to fuzzy logic systems and provides linguistic meaning to the connectionist architectures. Heuristic fuzzy logic rules and input-output fuzzy membership functions can be optimally tuned from training examples by a hybrid learning scheme comprised of two phases: rule generation phase from data; and rule tuning phase using error backpropagation learning scheme for a neural fuzzy system. To illustrate the performance and applicability of the proposed neuro-fuzzy hybrid model, extensive simulation studies of nonlinear complex dynamic systems are carried out. The proposed method can be applied to an on-line incremental adaptive learning for the prediction and control of nonlinear dynamical systems. Two benchmark case studies are used to demonstrate that the proposed HyFIS system is a superior neuro-fuzzy modelling technique.

Cognitive control predicts use of model-based reinforcement learning.

PubMed

Otto, A Ross; Skatova, Anya; Madlon-Kay, Seth; Daw, Nathaniel D

2015-02-01

Accounts of decision-making and its neural substrates have long posited the operation of separate, competing valuation systems in the control of choice behavior. Recent theoretical and experimental work suggest that this classic distinction between behaviorally and neurally dissociable systems for habitual and goal-directed (or more generally, automatic and controlled) choice may arise from two computational strategies for reinforcement learning (RL), called model-free and model-based RL, but the cognitive or computational processes by which one system may dominate over the other in the control of behavior is a matter of ongoing investigation. To elucidate this question, we leverage the theoretical framework of cognitive control, demonstrating that individual differences in utilization of goal-related contextual information--in the service of overcoming habitual, stimulus-driven responses--in established cognitive control paradigms predict model-based behavior in a separate, sequential choice task. The behavioral correspondence between cognitive control and model-based RL compellingly suggests that a common set of processes may underpin the two behaviors. In particular, computational mechanisms originally proposed to underlie controlled behavior may be applicable to understanding the interactions between model-based and model-free choice behavior.

Real time eye tracking using Kalman extended spatio-temporal context learning

NASA Astrophysics Data System (ADS)

Munir, Farzeen; Minhas, Fayyaz ul Amir Asfar; Jalil, Abdul; Jeon, Moongu

2017-06-01

Real time eye tracking has numerous applications in human computer interaction such as a mouse cursor control in a computer system. It is useful for persons with muscular or motion impairments. However, tracking the movement of the eye is complicated by occlusion due to blinking, head movement, screen glare, rapid eye movements, etc. In this work, we present the algorithmic and construction details of a real time eye tracking system. Our proposed system is an extension of Spatio-Temporal context learning through Kalman Filtering. Spatio-Temporal Context Learning offers state of the art accuracy in general object tracking but its performance suffers due to object occlusion. Addition of the Kalman filter allows the proposed method to model the dynamics of the motion of the eye and provide robust eye tracking in cases of occlusion. We demonstrate the effectiveness of this tracking technique by controlling the computer cursor in real time by eye movements.

Off-Policy Integral Reinforcement Learning Method to Solve Nonlinear Continuous-Time Multiplayer Nonzero-Sum Games.

PubMed

Song, Ruizhuo; Lewis, Frank L; Wei, Qinglai

2017-03-01

This paper establishes an off-policy integral reinforcement learning (IRL) method to solve nonlinear continuous-time (CT) nonzero-sum (NZS) games with unknown system dynamics. The IRL algorithm is presented to obtain the iterative control and off-policy learning is used to allow the dynamics to be completely unknown. Off-policy IRL is designed to do policy evaluation and policy improvement in the policy iteration algorithm. Critic and action networks are used to obtain the performance index and control for each player. The gradient descent algorithm makes the update of critic and action weights simultaneously. The convergence analysis of the weights is given. The asymptotic stability of the closed-loop system and the existence of Nash equilibrium are proved. The simulation study demonstrates the effectiveness of the developed method for nonlinear CT NZS games with unknown system dynamics.

Selected Flight Test Results for Online Learning Neural Network-Based Flight Control System

NASA Technical Reports Server (NTRS)

Williams, Peggy S.

2004-01-01

The NASA F-15 Intelligent Flight Control System project team has developed a series of flight control concepts designed to demonstrate the benefits of a neural network-based adaptive controller. The objective of the team is to develop and flight-test control systems that use neural network technology to optimize the performance of the aircraft under nominal conditions as well as stabilize the aircraft under failure conditions. Failure conditions include locked or failed control surfaces as well as unforeseen damage that might occur to the aircraft in flight. This report presents flight-test results for an adaptive controller using stability and control derivative values from an online learning neural network. A dynamic cell structure neural network is used in conjunction with a real-time parameter identification algorithm to estimate aerodynamic stability and control derivative increments to the baseline aerodynamic derivatives in flight. This set of open-loop flight tests was performed in preparation for a future phase of flights in which the learning neural network and parameter identification algorithm output would provide the flight controller with aerodynamic stability and control derivative updates in near real time. Two flight maneuvers are analyzed a pitch frequency sweep and an automated flight-test maneuver designed to optimally excite the parameter identification algorithm in all axes. Frequency responses generated from flight data are compared to those obtained from nonlinear simulation runs. An examination of flight data shows that addition of the flight-identified aerodynamic derivative increments into the simulation improved the pitch handling qualities of the aircraft.

Overlay improvements using a real time machine learning algorithm

NASA Astrophysics Data System (ADS)

Schmitt-Weaver, Emil; Kubis, Michael; Henke, Wolfgang; Slotboom, Daan; Hoogenboom, Tom; Mulkens, Jan; Coogans, Martyn; ten Berge, Peter; Verkleij, Dick; van de Mast, Frank

2014-04-01

While semiconductor manufacturing is moving towards the 14nm node using immersion lithography, the overlay requirements are tightened to below 5nm. Next to improvements in the immersion scanner platform, enhancements in the overlay optimization and process control are needed to enable these low overlay numbers. Whereas conventional overlay control methods address wafer and lot variation autonomously with wafer pre exposure alignment metrology and post exposure overlay metrology, we see a need to reduce these variations by correlating more of the TWINSCAN system's sensor data directly to the post exposure YieldStar metrology in time. In this paper we will present the results of a study on applying a real time control algorithm based on machine learning technology. Machine learning methods use context and TWINSCAN system sensor data paired with post exposure YieldStar metrology to recognize generic behavior and train the control system to anticipate on this generic behavior. Specific for this study, the data concerns immersion scanner context, sensor data and on-wafer measured overlay data. By making the link between the scanner data and the wafer data we are able to establish a real time relationship. The result is an inline controller that accounts for small changes in scanner hardware performance in time while picking up subtle lot to lot and wafer to wafer deviations introduced by wafer processing.

Thermal Control Technologies for Complex Spacecraft

NASA Technical Reports Server (NTRS)

Swanson, Theodore D.

2004-01-01

Thermal control is a generic need for all spacecraft. In response to ever more demanding science and exploration requirements, spacecraft are becoming ever more complex, and hence their thermal control systems must evolve. This paper briefly discusses the process of technology development, the state-of-the-art in thermal control, recent experiences with on-orbit two-phase systems, and the emerging thermal control technologies to meet these evolving needs. Some "lessons learned" based on experience with on-orbit systems are also presented.

A generalised optimal linear quadratic tracker with universal applications. Part 2: discrete-time systems

NASA Astrophysics Data System (ADS)

Ebrahimzadeh, Faezeh; Tsai, Jason Sheng-Hong; Chung, Min-Ching; Liao, Ying Ting; Guo, Shu-Mei; Shieh, Leang-San; Wang, Li

2017-01-01

Contrastive to Part 1, Part 2 presents a generalised optimal linear quadratic digital tracker (LQDT) with universal applications for the discrete-time (DT) systems. This includes (1) a generalised optimal LQDT design for the system with the pre-specified trajectories of the output and the control input and additionally with both the input-to-output direct-feedthrough term and known/estimated system disturbances or extra input/output signals; (2) a new optimal filter-shaped proportional plus integral state-feedback LQDT design for non-square non-minimum phase DT systems to achieve a minimum-phase-like tracking performance; (3) a new approach for computing the control zeros of the given non-square DT systems; and (4) a one-learning-epoch input-constrained iterative learning LQDT design for the repetitive DT systems.

Reinforcement learning controller design for affine nonlinear discrete-time systems using online approximators.

PubMed

Yang, Qinmin; Jagannathan, Sarangapani

2012-04-01

In this paper, reinforcement learning state- and output-feedback-based adaptive critic controller designs are proposed by using the online approximators (OLAs) for a general multi-input and multioutput affine unknown nonlinear discretetime systems in the presence of bounded disturbances. The proposed controller design has two entities, an action network that is designed to produce optimal signal and a critic network that evaluates the performance of the action network. The critic estimates the cost-to-go function which is tuned online using recursive equations derived from heuristic dynamic programming. Here, neural networks (NNs) are used both for the action and critic whereas any OLAs, such as radial basis functions, splines, fuzzy logic, etc., can be utilized. For the output-feedback counterpart, an additional NN is designated as the observer to estimate the unavailable system states, and thus, separation principle is not required. The NN weight tuning laws for the controller schemes are also derived while ensuring uniform ultimate boundedness of the closed-loop system using Lyapunov theory. Finally, the effectiveness of the two controllers is tested in simulation on a pendulum balancing system and a two-link robotic arm system.

Central Nervous System Control of Voice and Swallowing

PubMed Central

Ludlow, Christy L.

2015-01-01

This review of the central nervous control systems for voice and swallowing has suggested that the traditional concepts of a separation between cortical and limbic and brain stem control should be refined and more integrative. For voice production, a separation of the non-human vocalization system from the human learned voice production system has been posited based primarily on studies of non-human primates. However, recent humans studies of emotionally based vocalizations and human volitional voice production has shown more integration between these two systems than previously proposed. Recent human studies have shown that reflexive vocalization as well as learned voice production not involving speech, involve a common integrative system. On the other hand, recent studies of non-human primates have provided evidence of some cortical activity during vocalization and cortical changes with training during vocal behavior. For swallowing, evidence from the macaque and functional brain imaging in humans indicates that the control for the pharyngeal phase of swallowing is not primarily under brain stem mechanisms as previously proposed. Studies suggest that the initiation and patterning of swallowing for the pharyngeal phase is also under active cortical control for both spontaneous as well as volitional swallowing in awake humans and non-human primates. PMID:26241238

Technology and Education: Partners in Excellence.

ERIC Educational Resources Information Center

Main, Christine; Berry, Marsha

1993-01-01

Northern Kentucky University uses Ole, online learning educational system, which provides an interactive learning environment. Pilot participants received better grades and evaluated instructors more favorably than controls. An average of 30 percent savings per student over five years is expected. (SK)

A Comparison of Learning Environments: All That Glitters... Interim Technical Paper for Period January 1990-July 1991.

ERIC Educational Resources Information Center

Shute, Valerie J.

Aptitude-treatment interactions (ATIs) refer to the covariation between learner characteristic and instructional treatment in relation to some outcome measure. To systematically test for ATI, an intelligent tutoring system instructing in basic principles of electricity was chosen as a complex but controlled learning task. Two learning environments…

Open Learning: Resource Allocation and Control. Coombe Lodge Working Paper. Information Bank Number 1610.

ERIC Educational Resources Information Center

Latcham, Jack

In an effort to demonstrate that open learning can be accommodated within existing funding mechanisms in British colleges of further and higher education, this paper discusses issues concerned with determining the cost of Flexistudy and open learning delivery systems. After a review of the difficulties of estimating precisely the costs of various…

Evaluating Listening and Speaking Skills in a Mobile Game-Based Learning Environment with Situational Contexts

ERIC Educational Resources Information Center

Hwang, Wu-Yuin; Shih, Timothy K.; Ma, Zhao-Heng; Shadiev, Rustam; Chen, Shu-Yu

2016-01-01

Game-based learning activities that facilitate students' listening and speaking skills were designed in this study. To participate in learning activities, students in the control group used traditional methods, while students in the experimental group used a mobile system. In our study, we looked into the feasibility of mobile game-based learning…

The Perplexing Role of Learner Control in E-Learning: Will Learning and Transfer Benefit or Suffer?

ERIC Educational Resources Information Center

Granger, Benjamin P.; Levine, Edward L.

2010-01-01

The appeal of e-learning is not surprising given its many proposed advantages (e.g. flexibility, responsiveness to trainees' individual needs, potential cost-effectiveness). However, as pointed out by Ruel et al., academic research in support of the proposed advantages of electronic human resource management systems is scarce. Although this is…

Wiki and Digital Video Use in Strategic Interaction-Based Experiential EFL Learning

ERIC Educational Resources Information Center

Dehaan, Jonathan; Johnson, Neil H.; Yoshimura, Noriko; Kondo, Takako

2012-01-01

This paper details the use of a free and access-controlled wiki as the learning management system for a four-week teaching module designed to improve the oral communication skills of Japanese university EFL students. Students engaged in repeated experiential learning cycles of planning, doing, observing, and evaluating their performance of a role…

Neuromodulatory adaptive combination of correlation-based learning in cerebellum and reward-based learning in basal ganglia for goal-directed behavior control

PubMed Central

Dasgupta, Sakyasingha; Wörgötter, Florentin; Manoonpong, Poramate

2014-01-01

Goal-directed decision making in biological systems is broadly based on associations between conditional and unconditional stimuli. This can be further classified as classical conditioning (correlation-based learning) and operant conditioning (reward-based learning). A number of computational and experimental studies have well established the role of the basal ganglia in reward-based learning, where as the cerebellum plays an important role in developing specific conditioned responses. Although viewed as distinct learning systems, recent animal experiments point toward their complementary role in behavioral learning, and also show the existence of substantial two-way communication between these two brain structures. Based on this notion of co-operative learning, in this paper we hypothesize that the basal ganglia and cerebellar learning systems work in parallel and interact with each other. We envision that such an interaction is influenced by reward modulated heterosynaptic plasticity (RMHP) rule at the thalamus, guiding the overall goal directed behavior. Using a recurrent neural network actor-critic model of the basal ganglia and a feed-forward correlation-based learning model of the cerebellum, we demonstrate that the RMHP rule can effectively balance the outcomes of the two learning systems. This is tested using simulated environments of increasing complexity with a four-wheeled robot in a foraging task in both static and dynamic configurations. Although modeled with a simplified level of biological abstraction, we clearly demonstrate that such a RMHP induced combinatorial learning mechanism, leads to stabler and faster learning of goal-directed behaviors, in comparison to the individual systems. Thus, in this paper we provide a computational model for adaptive combination of the basal ganglia and cerebellum learning systems by way of neuromodulated plasticity for goal-directed decision making in biological and bio-mimetic organisms. PMID:25389391

Neuromodulatory adaptive combination of correlation-based learning in cerebellum and reward-based learning in basal ganglia for goal-directed behavior control.

PubMed

Dasgupta, Sakyasingha; Wörgötter, Florentin; Manoonpong, Poramate

2014-01-01

Goal-directed decision making in biological systems is broadly based on associations between conditional and unconditional stimuli. This can be further classified as classical conditioning (correlation-based learning) and operant conditioning (reward-based learning). A number of computational and experimental studies have well established the role of the basal ganglia in reward-based learning, where as the cerebellum plays an important role in developing specific conditioned responses. Although viewed as distinct learning systems, recent animal experiments point toward their complementary role in behavioral learning, and also show the existence of substantial two-way communication between these two brain structures. Based on this notion of co-operative learning, in this paper we hypothesize that the basal ganglia and cerebellar learning systems work in parallel and interact with each other. We envision that such an interaction is influenced by reward modulated heterosynaptic plasticity (RMHP) rule at the thalamus, guiding the overall goal directed behavior. Using a recurrent neural network actor-critic model of the basal ganglia and a feed-forward correlation-based learning model of the cerebellum, we demonstrate that the RMHP rule can effectively balance the outcomes of the two learning systems. This is tested using simulated environments of increasing complexity with a four-wheeled robot in a foraging task in both static and dynamic configurations. Although modeled with a simplified level of biological abstraction, we clearly demonstrate that such a RMHP induced combinatorial learning mechanism, leads to stabler and faster learning of goal-directed behaviors, in comparison to the individual systems. Thus, in this paper we provide a computational model for adaptive combination of the basal ganglia and cerebellum learning systems by way of neuromodulated plasticity for goal-directed decision making in biological and bio-mimetic organisms.

Relationships of Cognitive and Metacognitive Learning Strategies to Mathematics Achievement in Four High-Performing East Asian Education Systems

ERIC Educational Resources Information Center

Areepattamannil, Shaljan; Caleon, Imelda S.

2013-01-01

The authors examined the relationships of cognitive (i.e., memorization and elaboration) and metacognitive learning strategies (i.e., control strategies) to mathematics achievement among 15-year-old students in 4 high-performing East Asian education systems: Shanghai-China, Hong Kong-China, Korea, and Singapore. In all 4 East Asian education…

Inclusion of Immersive Virtual Learning Environments and Visual Control Systems to Support the Learning of Students with Asperger Syndrome

ERIC Educational Resources Information Center

Lorenzo, Gonzalo; Pomares, Jorge; Lledo, Asuncion

2013-01-01

This paper presents the use of immersive virtual reality systems in the educational intervention with Asperger students. The starting points of this study are features of these students' cognitive style that requires an explicit teaching style supported by visual aids and highly structured environments. The proposed immersive virtual reality…

A Development of Game-Based Learning Environment to Activate Interaction among Learners

NASA Astrophysics Data System (ADS)

Takaoka, Ryo; Shimokawa, Masayuki; Okamoto, Toshio

Many studies and systems that incorporate elements such as “pleasure” and “fun” in the game to improve a learner's motivation have been developed in the field of learning environments. However, few are the studies of situations where many learners gather at a single computer and participate in a game-based learning environment (GBLE), and where the GBLE designs the learning process by controlling the interactions between learners such as competition, collaboration, and learning by teaching. Therefore, the purpose of this study is to propose a framework of educational control that induces and activates interaction between learners intentionally to create a learning opportunity that is based on the knowledge understanding model of each learner. In this paper, we explain the design philosophy and the framework of our GBLE called “Who becomes the king in the country of mathematics?” from a game viewpoint and describe the method of learning support control in the learning environment. In addition, we report the results of the learning experiment with our GBLE, which we carried out in a junior high school, and include some comments by a principal and a teacher. From the results of the experiment and some comments, we noticed that a game may play a significant role in weakening the learning relationship among students and creating new relationships in the world of the game. Furthermore, we discovered that learning support control of the GBLE has led to activation of the interaction between learners to some extent.

Theories and control models and motor learning: clinical applications in neuro-rehabilitation.

PubMed

Cano-de-la-Cuerda, R; Molero-Sánchez, A; Carratalá-Tejada, M; Alguacil-Diego, I M; Molina-Rueda, F; Miangolarra-Page, J C; Torricelli, D

2015-01-01

In recent decades there has been a special interest in theories that could explain the regulation of motor control, and their applications. These theories are often based on models of brain function, philosophically reflecting different criteria on how movement is controlled by the brain, each being emphasised in different neural components of the movement. The concept of motor learning, regarded as the set of internal processes associated with practice and experience that produce relatively permanent changes in the ability to produce motor activities through a specific skill, is also relevant in the context of neuroscience. Thus, both motor control and learning are seen as key fields of study for health professionals in the field of neuro-rehabilitation. The major theories of motor control are described, which include, motor programming theory, systems theory, the theory of dynamic action, and the theory of parallel distributed processing, as well as the factors that influence motor learning and its applications in neuro-rehabilitation. At present there is no consensus on which theory or model defines the regulations to explain motor control. Theories of motor learning should be the basis for motor rehabilitation. The new research should apply the knowledge generated in the fields of control and motor learning in neuro-rehabilitation. Copyright © 2011 Sociedad Española de Neurología. Published by Elsevier Espana. All rights reserved.

Iterative learning control with applications in energy generation, lasers and health care

PubMed Central

Tutty, O. R.

2016-01-01

Many physical systems make repeated executions of the same finite time duration task. One example is a robot in a factory or warehouse whose task is to collect an object in sequence from a location, transfer it over a finite duration, place it at a specified location or on a moving conveyor and then return for the next one and so on. Iterative learning control was especially developed for systems with this mode of operation and this paper gives an overview of this control design method using relatively recent relevant applications in wind turbines, free-electron lasers and health care, as exemplars to demonstrate its applicability. PMID:27713654

Normal Aging and the Dissociable Prototype Learning Systems

PubMed Central

Glass, Brian D.; Chotibut, Tanya; Pacheco, Jennifer; Schnyer, David M.; Maddox, W. Todd

2011-01-01

Dissociable prototype learning systems have been demonstrated behaviorally and with neuroimaging in younger adults as well as with patient populations. In A/not-A (AN) prototype learning, participants are shown members of category A during training, and during test are asked to decide whether novel items are in category A or are not in category A. Research suggests that AN learning is mediated by a perceptual learning system. In A/B (AB) prototype learning, participants are shown members of category A and B during training, and during test are asked to decide whether novel items are in category A or category B. In contrast to AN, research suggests that AB learning is mediated by a declarative memory system. The current study examined the effects of normal aging on AN and AB prototype learning. We observed an age-related deficit in AB learning, but an age-related advantage in AN learning. Computational modeling supports one possible interpretation based on narrower selective attentional focus in older adults in the AB task and broader selective attention in the AN task. Neuropsychological testing in older participants suggested that executive functioning and attentional control were associated with better performance in both tasks. However, nonverbal memory was associated with better AN performance, while visual attention was associated with worse AB performance. The results support an interactive memory systems approach and suggest that age-related declines in one memory system can lead to deficits in some tasks, but to enhanced performance in others. PMID:21875215

Adaptive robotic control driven by a versatile spiking cerebellar network.

PubMed

Casellato, Claudia; Antonietti, Alberto; Garrido, Jesus A; Carrillo, Richard R; Luque, Niceto R; Ros, Eduardo; Pedrocchi, Alessandra; D'Angelo, Egidio

2014-01-01

The cerebellum is involved in a large number of different neural processes, especially in associative learning and in fine motor control. To develop a comprehensive theory of sensorimotor learning and control, it is crucial to determine the neural basis of coding and plasticity embedded into the cerebellar neural circuit and how they are translated into behavioral outcomes in learning paradigms. Learning has to be inferred from the interaction of an embodied system with its real environment, and the same cerebellar principles derived from cell physiology have to be able to drive a variety of tasks of different nature, calling for complex timing and movement patterns. We have coupled a realistic cerebellar spiking neural network (SNN) with a real robot and challenged it in multiple diverse sensorimotor tasks. Encoding and decoding strategies based on neuronal firing rates were applied. Adaptive motor control protocols with acquisition and extinction phases have been designed and tested, including an associative Pavlovian task (Eye blinking classical conditioning), a vestibulo-ocular task and a perturbed arm reaching task operating in closed-loop. The SNN processed in real-time mossy fiber inputs as arbitrary contextual signals, irrespective of whether they conveyed a tone, a vestibular stimulus or the position of a limb. A bidirectional long-term plasticity rule implemented at parallel fibers-Purkinje cell synapses modulated the output activity in the deep cerebellar nuclei. In all tasks, the neurorobot learned to adjust timing and gain of the motor responses by tuning its output discharge. It succeeded in reproducing how human biological systems acquire, extinguish and express knowledge of a noisy and changing world. By varying stimuli and perturbations patterns, real-time control robustness and generalizability were validated. The implicit spiking dynamics of the cerebellar model fulfill timing, prediction and learning functions.

Reinforcement-learning-based dual-control methodology for complex nonlinear discrete-time systems with application to spark engine EGR operation.

PubMed

Shih, Peter; Kaul, Brian C; Jagannathan, S; Drallmeier, James A

2008-08-01

A novel reinforcement-learning-based dual-control methodology adaptive neural network (NN) controller is developed to deliver a desired tracking performance for a class of complex feedback nonlinear discrete-time systems, which consists of a second-order nonlinear discrete-time system in nonstrict feedback form and an affine nonlinear discrete-time system, in the presence of bounded and unknown disturbances. For example, the exhaust gas recirculation (EGR) operation of a spark ignition (SI) engine is modeled by using such a complex nonlinear discrete-time system. A dual-controller approach is undertaken where primary adaptive critic NN controller is designed for the nonstrict feedback nonlinear discrete-time system whereas the secondary one for the affine nonlinear discrete-time system but the controllers together offer the desired performance. The primary adaptive critic NN controller includes an NN observer for estimating the states and output, an NN critic, and two action NNs for generating virtual control and actual control inputs for the nonstrict feedback nonlinear discrete-time system, whereas an additional critic NN and an action NN are included for the affine nonlinear discrete-time system by assuming the state availability. All NN weights adapt online towards minimization of a certain performance index, utilizing gradient-descent-based rule. Using Lyapunov theory, the uniformly ultimate boundedness (UUB) of the closed-loop tracking error, weight estimates, and observer estimates are shown. The adaptive critic NN controller performance is evaluated on an SI engine operating with high EGR levels where the controller objective is to reduce cyclic dispersion in heat release while minimizing fuel intake. Simulation and experimental results indicate that engine out emissions drop significantly at 20% EGR due to reduction in dispersion in heat release thus verifying the dual-control approach.

Measuring Engagement as Students Learn Dynamic Systems and Control with a Video Game

ERIC Educational Resources Information Center

Coller, B. D.; Shernoff, David J.; Strati, Anna

2011-01-01

The paper presents results of a multi-year quasi-experimental study of student engagement during which a video game was introduced into an undergraduate dynamic systems and control course. The video game, "EduTorcs", provided challenges in which students devised control algorithms that drive virtual cars and ride virtual bikes through a…

Drinking Water Distribution Systems

EPA Pesticide Factsheets

Learn about an overview of drinking water distribution systems, the factors that degrade water quality in the distribution system, assessments of risk, future research about these risks, and how to reduce cross-connection control risk.

Design, Integration, Certification and Testing of the Orion Crew Module Propulsion System

NASA Technical Reports Server (NTRS)

McKay, Heather; Coffman, Eric; May, Sarah; Freeman, Rich; Cain, George; Albright, John; Schoenberg, Rich; Delventhal, Rex

2014-01-01

The Orion Crew Module Propulsion Reaction Control System is currently complete and ready for flight as part of the Orion program's first flight test, Exploration Flight Test One (EFT-1). As part of the first article design, build, test, and integration effort, several key lessons learned have been noted and are planned for incorporation into the next build of the system. This paper provides an overview of those lessons learned and a status on the Orion propulsion system progress to date.

Learning to push and learning to move: the adaptive control of contact forces

PubMed Central

Casadio, Maura; Pressman, Assaf; Mussa-Ivaldi, Ferdinando A.

2015-01-01

To be successful at manipulating objects one needs to apply simultaneously well controlled movements and contact forces. We present a computational theory of how the brain may successfully generate a vast spectrum of interactive behaviors by combining two independent processes. One process is competent to control movements in free space and the other is competent to control contact forces against rigid constraints. Free space and rigid constraints are singularities at the boundaries of a continuum of mechanical impedance. Within this continuum, forces and motions occur in “compatible pairs” connected by the equations of Newtonian dynamics. The force applied to an object determines its motion. Conversely, inverse dynamics determine a unique force trajectory from a movement trajectory. In this perspective, we describe motor learning as a process leading to the discovery of compatible force/motion pairs. The learned compatible pairs constitute a local representation of the environment's mechanics. Experiments on force field adaptation have already provided us with evidence that the brain is able to predict and compensate the forces encountered when one is attempting to generate a motion. Here, we tested the theory in the dual case, i.e., when one attempts at applying a desired contact force against a simulated rigid surface. If the surface becomes unexpectedly compliant, the contact point moves as a function of the applied force and this causes the applied force to deviate from its desired value. We found that, through repeated attempts at generating the desired contact force, subjects discovered the unique compatible hand motion. When, after learning, the rigid contact was unexpectedly restored, subjects displayed after effects of learning, consistent with the concurrent operation of a motion control system and a force control system. Together, theory and experiment support a new and broader view of modularity in the coordinated control of forces and motions. PMID:26594163

Flyback CCM inverter for AC module applications: iterative learning control and convergence analysis

NASA Astrophysics Data System (ADS)

Lee, Sung-Ho; Kim, Minsung

2017-12-01

This paper presents an iterative learning controller (ILC) for an interleaved flyback inverter operating in continuous conduction mode (CCM). The flyback CCM inverter features small output ripple current, high efficiency, and low cost, and hence it is well suited for photovoltaic power applications. However, it exhibits the non-minimum phase behaviour, because its transfer function from control duty to output current has the right-half-plane (RHP) zero. Moreover, the flyback CCM inverter suffers from the time-varying grid voltage disturbance. Thus, conventional control scheme results in inaccurate output tracking. To overcome these problems, the ILC is first developed and applied to the flyback inverter operating in CCM. The ILC makes use of both predictive and current learning terms which help the system output to converge to the reference trajectory. We take into account the nonlinear averaged model and use it to construct the proposed controller. It is proven that the system output globally converges to the reference trajectory in the absence of state disturbances, output noises, or initial state errors. Numerical simulations are performed to validate the proposed control scheme, and experiments using 400-W AC module prototype are carried out to demonstrate its practical feasibility.

Consciousness: the radical plasticity thesis.

PubMed

Cleeremans, Axel

2008-01-01

In this chapter, I sketch a conceptual framework which takes it as a starting point that conscious and unconscious cognition are rooted in the same set of interacting learning mechanisms and representational systems. On this view, the extent to which a representation is conscious depends in a graded manner on properties such as its stability in time or its strength. Crucially, these properties are accrued as a result of learning, which is in turn viewed as a mandatory process that always accompanies information processing. From this perspective, consciousness is best characterized as involving (1) a graded continuum defined over "quality of representation", such that availability to consciousness and to cognitive control correlates with quality, and (2) the implication of systems of metarepresentations. A first implication of these ideas is that the main function of consciousness is to make flexible, adaptive control over behavior possible. A second, much more speculative implication, is that we learn to be conscious. This I call the "radical plasticity thesis"--the hypothesis that consciousness emerges in systems capable not only of learning about their environment, but also about their own internal representations of it.

Drosophila learn efficient paths to a food source.

PubMed

Navawongse, Rapeechai; Choudhury, Deepak; Raczkowska, Marlena; Stewart, James Charles; Lim, Terrence; Rahman, Mashiur; Toh, Alicia Guek Geok; Wang, Zhiping; Claridge-Chang, Adam

2016-05-01

Elucidating the genetic, and neuronal bases for learned behavior is a central problem in neuroscience. A leading system for neurogenetic discovery is the vinegar fly Drosophila melanogaster; fly memory research has identified genes and circuits that mediate aversive and appetitive learning. However, methods to study adaptive food-seeking behavior in this animal have lagged decades behind rodent feeding analysis, largely due to the challenges presented by their small scale. There is currently no method to dynamically control flies' access to food. In rodents, protocols that use dynamic food delivery are a central element of experimental paradigms that date back to the influential work of Skinner. This method is still commonly used in the analysis of learning, memory, addiction, feeding, and many other subjects in experimental psychology. The difficulty of microscale food delivery means this is not a technique used in fly behavior. In the present manuscript we describe a microfluidic chip integrated with machine vision and automation to dynamically control defined liquid food presentations and sensory stimuli. Strikingly, repeated presentations of food at a fixed location produced improvements in path efficiency during food approach. This shows that improved path choice is a learned behavior. Active control of food availability using this microfluidic system is a valuable addition to the methods currently available for the analysis of learned feeding behavior in flies. Copyright © 2016 The Authors. Published by Elsevier Inc. All rights reserved.

Prototype learning and dissociable categorization systems in Alzheimer's disease.

PubMed

Heindel, William C; Festa, Elena K; Ott, Brian R; Landy, Kelly M; Salmon, David P

2013-08-01

Recent neuroimaging studies suggest that prototype learning may be mediated by at least two dissociable memory systems depending on the mode of acquisition, with A/Not-A prototype learning dependent upon a perceptual representation system located within posterior visual cortex and A/B prototype learning dependent upon a declarative memory system associated with medial temporal and frontal regions. The degree to which patients with Alzheimer's disease (AD) can acquire new categorical information may therefore critically depend upon the mode of acquisition. The present study examined A/Not-A and A/B prototype learning in AD patients using procedures that allowed direct comparison of learning across tasks. Despite impaired explicit recall of category features in all tasks, patients showed differential patterns of category acquisition across tasks. First, AD patients demonstrated impaired prototype induction along with intact exemplar classification under incidental A/Not-A conditions, suggesting that the loss of functional connectivity within visual cortical areas disrupted the integration processes supporting prototype induction within the perceptual representation system. Second, AD patients demonstrated intact prototype induction but impaired exemplar classification during A/B learning under observational conditions, suggesting that this form of prototype learning is dependent on a declarative memory system that is disrupted in AD. Third, the surprisingly intact classification of both prototypes and exemplars during A/B learning under trial-and-error feedback conditions suggests that AD patients shifted control from their deficient declarative memory system to a feedback-dependent procedural memory system when training conditions allowed. Taken together, these findings serve to not only increase our understanding of category learning in AD, but to also provide new insights into the ways in which different memory systems interact to support the acquisition of categorical knowledge. Copyright © 2013 Elsevier Ltd. All rights reserved.

Prototype Learning and Dissociable Categorization Systems in Alzheimer’s Disease

PubMed Central

Heindel, William C.; Festa, Elena K.; Ott, Brian R.; Landy, Kelly M.; Salmon, David P.

2015-01-01

Recent neuroimaging studies suggest that prototype learning may be mediated by at least two dissociable memory systems depending on the mode of acquisition, with A/Not-A prototype learning dependent upon a perceptual representation system located within posterior visual cortex and A/B prototype learning dependent upon a declarative memory system associated with medial temporal and frontal regions. The degree to which patients with Alzheimer’s disease (AD) can acquire new categorical information may therefore critically depend upon the mode of acquisition. The present study examined A/Not-A and A/B prototype learning in AD patients using procedures that allowed direct comparison of learning across tasks. Despite impaired explicit recall of category features in all tasks, patients showed differential patterns of category acquisition across tasks. First, AD patients demonstrated impaired prototype induction along with intact exemplar classification under incidental A/Not-A conditions, suggesting that the loss of functional connectivity within visual cortical areas disrupted the integration processes supporting prototype induction within the perceptual representation system. Second, AD patients demonstrated intact prototype induction but impaired exemplar classification during A/B learning under observational conditions, suggesting that this form of prototype learning is dependent on a declarative memory system that is disrupted in AD. Third, the surprisingly intact classification of both prototypes and exemplars during A/B learning under trial-and-error feedback conditions suggests that AD patients shifted control from their deficient declarative memory system to a feedback-dependent procedural memory system when training conditions allowed. Taken together, these findings serve to not only increase our understanding of category learning in AD, but to also provide new insights into the ways in which different memory systems interact to support the acquisition of categorical knowledge. PMID:23751172

Autonomous learning based on cost assumptions: theoretical studies and experiments in robot control.

PubMed

Ribeiro, C H; Hemerly, E M

2000-02-01

Autonomous learning techniques are based on experience acquisition. In most realistic applications, experience is time-consuming: it implies sensor reading, actuator control and algorithmic update, constrained by the learning system dynamics. The information crudeness upon which classical learning algorithms operate make such problems too difficult and unrealistic. Nonetheless, additional information for facilitating the learning process ideally should be embedded in such a way that the structural, well-studied characteristics of these fundamental algorithms are maintained. We investigate in this article a more general formulation of the Q-learning method that allows for a spreading of information derived from single updates towards a neighbourhood of the instantly visited state and converges to optimality. We show how this new formulation can be used as a mechanism to safely embed prior knowledge about the structure of the state space, and demonstrate it in a modified implementation of a reinforcement learning algorithm in a real robot navigation task.

The relationship of neurogenesis and growth of brain regions to song learning

PubMed Central

Kirn, John R.

2009-01-01

Song learning, maintenance and production require coordinated activity across multiple auditory, sensory-motor, and neuromuscular structures. Telencephalic components of the sensory-motor circuitry are unique to avian species that engage in song learning. The song system shows protracted development that begins prior to hatching but continues well into adulthood. The staggered developmental timetable for construction of the song system provides clues of subsystems involved in specific stages of song learning and maintenance. Progressive events, including neurogenesis and song system growth, as well as regressive events such as apoptosis and synapse elimination, occur during periods of song learning and the transitions between stereotyped and variable song during both development and adulthood. There is clear evidence that gonadal steroids influence the development of song attributes and shape the underlying neural circuitry. Some aspects of song system development are influenced by sensory, motor and social experience, while other aspects of neural development appear to be experience-independent. Although there are species differences in the extent to which song learning continues into adulthood, growing evidence suggests that despite differences in learning trajectories, adult refinement of song motor control and song maintenance can require remarkable behavioral and neural flexibility reminiscent of sensory-motor learning. PMID:19853905

Artificial neural network implementation of a near-ideal error prediction controller

NASA Technical Reports Server (NTRS)

Mcvey, Eugene S.; Taylor, Lynore Denise

1992-01-01

A theory has been developed at the University of Virginia which explains the effects of including an ideal predictor in the forward loop of a linear error-sampled system. It has been shown that the presence of this ideal predictor tends to stabilize the class of systems considered. A prediction controller is merely a system which anticipates a signal or part of a signal before it actually occurs. It is understood that an exact prediction controller is physically unrealizable. However, in systems where the input tends to be repetitive or limited, (i.e., not random) near ideal prediction is possible. In order for the controller to act as a stability compensator, the predictor must be designed in a way that allows it to learn the expected error response of the system. In this way, an unstable system will become stable by including the predicted error in the system transfer function. Previous and current prediction controller include pattern recognition developments and fast-time simulation which are applicable to the analysis of linear sampled data type systems. The use of pattern recognition techniques, along with a template matching scheme, has been proposed as one realizable type of near-ideal prediction. Since many, if not most, systems are repeatedly subjected to similar inputs, it was proposed that an adaptive mechanism be used to 'learn' the correct predicted error response. Once the system has learned the response of all the expected inputs, it is necessary only to recognize the type of input with a template matching mechanism and then to use the correct predicted error to drive the system. Suggested here is an alternate approach to the realization of a near-ideal error prediction controller, one designed using Neural Networks. Neural Networks are good at recognizing patterns such as system responses, and the back-propagation architecture makes use of a template matching scheme. In using this type of error prediction, it is assumed that the system error responses be known for a particular input and modeled plant. These responses are used in the error prediction controller. An analysis was done on the general dynamic behavior that results from including a digital error predictor in a control loop and these were compared to those including the near-ideal Neural Network error predictor. This analysis was done for a second and third order system.

EOL-1, the Homolog of the Mammalian Dom3Z, Regulates Olfactory Learning in C. elegans

PubMed Central

Shen, Yu; Zhang, Jiangwen; Calarco, John A.

2014-01-01

Learning is an essential function of the nervous system. However, our understanding of molecular underpinnings of learning remains incomplete. Here, we characterize a conserved protein EOL-1 that regulates olfactory learning in Caenorhabditis elegans. A recessive allele of eol-1 (enhanced olfactory learning) learns better to adjust its olfactory preference for bacteria foods and eol-1 acts in the URX sensory neurons to regulate learning. The mammalian homolog of EOL-1, Dom3Z, which regulates quality control of pre-mRNAs, can substitute the function of EOL-1 in learning regulation, demonstrating functional conservation between these homologs. Mutating the residues of Dom3Z that are critical for its enzymatic activity, and the equivalent residues in EOL-1, abolishes the function of these proteins in learning. Together, our results provide insights into the function of EOL-1/Dom3Z and suggest that its activity in pre-mRNA quality control is involved in neural plasticity. PMID:25274815

SCADA-based Operator Support System for Power Plant Equipment Fault Forecasting

NASA Astrophysics Data System (ADS)

Mayadevi, N.; Ushakumari, S. S.; Vinodchandra, S. S.

2014-12-01

Power plant equipment must be monitored closely to prevent failures from disrupting plant availability. Online monitoring technology integrated with hybrid forecasting techniques can be used to prevent plant equipment faults. A self learning rule-based expert system is proposed in this paper for fault forecasting in power plants controlled by supervisory control and data acquisition (SCADA) system. Self-learning utilizes associative data mining algorithms on the SCADA history database to form new rules that can dynamically update the knowledge base of the rule-based expert system. In this study, a number of popular associative learning algorithms are considered for rule formation. Data mining results show that the Tertius algorithm is best suited for developing a learning engine for power plants. For real-time monitoring of the plant condition, graphical models are constructed by K-means clustering. To build a time-series forecasting model, a multi layer preceptron (MLP) is used. Once created, the models are updated in the model library to provide an adaptive environment for the proposed system. Graphical user interface (GUI) illustrates the variation of all sensor values affecting a particular alarm/fault, as well as the step-by-step procedure for avoiding critical situations and consequent plant shutdown. The forecasting performance is evaluated by computing the mean absolute error and root mean square error of the predictions.

Control Systems with Normalized and Covariance Adaptation by Optimal Control Modification

NASA Technical Reports Server (NTRS)

Nguyen, Nhan T. (Inventor); Burken, John J. (Inventor); Hanson, Curtis E. (Inventor)

2016-01-01

Disclosed is a novel adaptive control method and system called optimal control modification with normalization and covariance adjustment. The invention addresses specifically to current challenges with adaptive control in these areas: 1) persistent excitation, 2) complex nonlinear input-output mapping, 3) large inputs and persistent learning, and 4) the lack of stability analysis tools for certification. The invention has been subject to many simulations and flight testing. The results substantiate the effectiveness of the invention and demonstrate the technical feasibility for use in modern aircraft flight control systems.

Beyond the Flipped Classroom: A Highly Interactive Cloud-Classroom (HIC) Embedded into Basic Materials Science Courses

NASA Astrophysics Data System (ADS)

Liou, Wei-Kai; Bhagat, Kaushal Kumar; Chang, Chun-Yen

2016-06-01

The present study compares the highly interactive cloud-classroom (HIC) system with traditional methods of teaching materials science that utilize crystal structure picture or real crystal structure model, in order to examine its learning effectiveness across three dimensions: knowledge, comprehension and application. The aim of this study was to evaluate the (HIC) system, which incorporates augmented reality, virtual reality and cloud-classroom to teach basic materials science courses. The study followed a pretest-posttest quasi-experimental research design. A total of 92 students (aged 19-20 years), in a second-year undergraduate program, participated in this 18-week-long experiment. The students were divided into an experimental group and a control group. The experimental group (36 males and 10 females) was instructed utilizing the HIC system, while the control group (34 males and 12 females) was led through traditional teaching methods. Pretest, posttest, and delayed posttest scores were evaluated by multivariate analysis of covariance. The results indicated that participants in the experimental group who used the HIC system outperformed the control group, in the both posttest and delayed posttest, across three learning dimensions. Based on these results, the HIC system is recommended to be incorporated in formal materials science learning settings.

Human factors aspects of control room design

NASA Technical Reports Server (NTRS)

Jenkins, J. P.

1983-01-01

A plan for the design and analysis of a multistation control room is reviewed. It is found that acceptance of the computer based information system by the uses in the control room is mandatory for mission and system success. Criteria to improve computer/user interface include: match of system input/output with user; reliability, compatibility and maintainability; easy to learn and little training needed; self descriptive system; system under user control; transparent language, format and organization; corresponds to user expectations; adaptable to user experience level; fault tolerant; dialog capability user communications needs reflected in flexibility, complexity, power and information load; integrated system; and documentation.

Dissociation of spatial memory systems in Williams syndrome.

PubMed

Bostelmann, Mathilde; Fragnière, Emilie; Costanzo, Floriana; Di Vara, Silvia; Menghini, Deny; Vicari, Stefano; Lavenex, Pierre; Lavenex, Pamela Banta

2017-11-01

Williams syndrome (WS), a genetic deletion syndrome, is characterized by severe visuospatial deficits affecting performance on both tabletop spatial tasks and on tasks which assess orientation and navigation. Nevertheless, previous studies of WS spatial capacities have ignored the fact that two different spatial memory systems are believed to contribute parallel spatial representations supporting navigation. The place learning system depends on the hippocampal formation and creates flexible relational representations of the environment, also known as cognitive maps. The spatial response learning system depends on the striatum and creates fixed stimulus-response representations, also known as habits. Indeed, no study assessing WS spatial competence has used tasks which selectively target these two spatial memory systems. Here, we report that individuals with WS exhibit a dissociation in their spatial abilities subserved by these two memory systems. As compared to typically developing (TD) children in the same mental age range, place learning performance was impaired in individuals with WS. In contrast, their spatial response learning performance was facilitated. Our findings in individuals with WS and TD children suggest that place learning and response learning interact competitively to control the behavioral strategies normally used to support human spatial navigation. Our findings further suggest that the neural pathways supporting place learning may be affected by the genetic deletion that characterizes WS, whereas those supporting response learning may be relatively preserved. The dissociation observed between these two spatial memory systems provides a coherent theoretical framework to characterize the spatial abilities of individuals with WS, and may lead to the development of new learning strategies based on their facilitated response learning abilities. © 2017 Wiley Periodicals, Inc.

Multi Agent Systems with Symbiotic Learning and Evolution using GNP

NASA Astrophysics Data System (ADS)

Eguchi, Toru; Hirasawa, Kotaro; Hu, Jinglu; Murata, Junichi

Recently, various attempts relevant to Multi Agent Systems (MAS) which is one of the most promising systems based on Distributed Artificial Intelligence have been studied to control large and complicated systems efficiently. In these trends of MAS, Multi Agent Systems with Symbiotic Learning and Evolution named Masbiole has been proposed. In Masbiole, symbiotic phenomena among creatures are considered in the process of learning and evolution of MAS. So we can expect more flexible and sophisticated solutions than conventional MAS. In this paper, we apply Masbiole to Iterative Prisoner’s Dilemma Games (IPD Games) using Genetic Network Programming (GNP) which is a newly developed evolutionary computation method for constituting agents. Some characteristics of Masbiole using GNP in IPD Games are clarified.

Systematic construction and control of stereo nerve vision network in intelligent manufacturing

NASA Astrophysics Data System (ADS)

Liu, Hua; Wang, Helong; Guo, Chunjie; Ding, Quanxin; Zhou, Liwei

2017-10-01

A system method of constructing stereo vision by using neural network is proposed, and the operation and control mechanism in actual operation are proposed. This method makes effective use of the neural network in learning and memory function, by after training with samples. Moreover, the neural network can learn the nonlinear relationship in the stereoscopic vision system and the internal and external orientation elements. These considerations are Worthy of attention, which includes limited constraints, the scientific of critical group, the operating speed and the operability in technical aspects. The results support our theoretical forecast.

Design & control of a 3D stroke rehabilitation platform.

PubMed

Cai, Z; Tong, D; Meadmore, K L; Freeman, C T; Hughes, A M; Rogers, E; Burridge, J H

2011-01-01

An upper limb stroke rehabilitation system is developed which combines electrical stimulation with mechanical arm support, to assist patients performing 3D reaching tasks in a virtual reality environment. The Stimulation Assistance through Iterative Learning (SAIL) platform applies electrical stimulation to two muscles in the arm using model-based control schemes which learn from previous trials of the task. This results in accurate movement which maximises the therapeutic effect of treatment. The principal components of the system are described and experimental results confirm its efficacy for clinical use in upper limb stroke rehabilitation. © 2011 IEEE

Integral reinforcement learning for continuous-time input-affine nonlinear systems with simultaneous invariant explorations.

PubMed

Lee, Jae Young; Park, Jin Bae; Choi, Yoon Ho

2015-05-01

This paper focuses on a class of reinforcement learning (RL) algorithms, named integral RL (I-RL), that solve continuous-time (CT) nonlinear optimal control problems with input-affine system dynamics. First, we extend the concepts of exploration, integral temporal difference, and invariant admissibility to the target CT nonlinear system that is governed by a control policy plus a probing signal called an exploration. Then, we show input-to-state stability (ISS) and invariant admissibility of the closed-loop systems with the policies generated by integral policy iteration (I-PI) or invariantly admissible PI (IA-PI) method. Based on these, three online I-RL algorithms named explorized I-PI and integral Q -learning I, II are proposed, all of which generate the same convergent sequences as I-PI and IA-PI under the required excitation condition on the exploration. All the proposed methods are partially or completely model free, and can simultaneously explore the state space in a stable manner during the online learning processes. ISS, invariant admissibility, and convergence properties of the proposed methods are also investigated, and related with these, we show the design principles of the exploration for safe learning. Neural-network-based implementation methods for the proposed schemes are also presented in this paper. Finally, several numerical simulations are carried out to verify the effectiveness of the proposed methods.

The organization of an autonomous learning system

NASA Technical Reports Server (NTRS)

Kanerva, Pentti

1988-01-01

The organization of systems that learn from experience is examined, human beings and animals being prime examples of such systems. How is their information processing organized. They build an internal model of the world and base their actions on the model. The model is dynamic and predictive, and it includes the systems' own actions and their effects. In modeling such systems, a large pattern of features represents a moment of the system's experience. Some of the features are provided by the system's senses, some control the system's motors, and the rest have no immediate external significance. A sequence of such patterns then represents the system's experience over time. By storing such sequences appropriately in memory, the system builds a world model based on experience. In addition to the essential function of memory, fundamental roles are played by a sensory system that makes raw information about the world suitable for memory storage and by a motor system that affects the world. The relation of sensory and motor systems to the memory is discussed, together with how favorable actions can be learned and unfavorable actions can be avoided. Results in classical learning theory are explained in terms of the model, more advanced forms of learning are discussed, and the relevance of the model to the frame problem of robotics is examined.

Reinforcement learning techniques for controlling resources in power networks

NASA Astrophysics Data System (ADS)

Kowli, Anupama Sunil

As power grids transition towards increased reliance on renewable generation, energy storage and demand response resources, an effective control architecture is required to harness the full functionalities of these resources. There is a critical need for control techniques that recognize the unique characteristics of the different resources and exploit the flexibility afforded by them to provide ancillary services to the grid. The work presented in this dissertation addresses these needs. Specifically, new algorithms are proposed, which allow control synthesis in settings wherein the precise distribution of the uncertainty and its temporal statistics are not known. These algorithms are based on recent developments in Markov decision theory, approximate dynamic programming and reinforcement learning. They impose minimal assumptions on the system model and allow the control to be "learned" based on the actual dynamics of the system. Furthermore, they can accommodate complex constraints such as capacity and ramping limits on generation resources, state-of-charge constraints on storage resources, comfort-related limitations on demand response resources and power flow limits on transmission lines. Numerical studies demonstrating applications of these algorithms to practical control problems in power systems are discussed. Results demonstrate how the proposed control algorithms can be used to improve the performance and reduce the computational complexity of the economic dispatch mechanism in a power network. We argue that the proposed algorithms are eminently suitable to develop operational decision-making tools for large power grids with many resources and many sources of uncertainty.

The remapping of space in motor learning and human-machine interfaces

PubMed Central

Mussa-Ivaldi, F.A.; Danziger, Z.

2009-01-01

Studies of motor adaptation to patterns of deterministic forces have revealed the ability of the motor control system to form and use predictive representations of the environment. One of the most fundamental elements of our environment is space itself. This article focuses on the notion of Euclidean space as it applies to common sensory motor experiences. Starting from the assumption that we interact with the world through a system of neural signals, we observe that these signals are not inherently endowed with metric properties of the ordinary Euclidean space. The ability of the nervous system to represent these properties depends on adaptive mechanisms that reconstruct the Euclidean metric from signals that are not Euclidean. Gaining access to these mechanisms will reveal the process by which the nervous system handles novel sophisticated coordinate transformation tasks, thus highlighting possible avenues to create functional human-machine interfaces that can make that task much easier. A set of experiments is presented that demonstrate the ability of the sensory-motor system to reorganize coordination in novel geometrical environments. In these environments multiple degrees of freedom of body motions are used to control the coordinates of a point in a two-dimensional Euclidean space. We discuss how practice leads to the acquisition of the metric properties of the controlled space. Methods of machine learning based on the reduction of reaching errors are tested as a means to facilitate learning by adaptively changing he map from body motions to controlled device. We discuss the relevance of the results to the development of adaptive human machine interfaces and optimal control. PMID:19665553

Effects of iterative learning based signal control strategies on macroscopic fundamental diagrams of urban road networks

NASA Astrophysics Data System (ADS)

Yan, Fei; Tian, Fuli; Shi, Zhongke

2016-10-01

Urban traffic flows are inherently repeated on a daily or weekly basis. This repeatability can help improve the traffic conditions if it is used properly by the control system. In this paper, we propose a novel iterative learning control (ILC) strategy for traffic signals of urban road networks using the repeatability feature of traffic flow. To improve the control robustness, the ILC strategy is further integrated with an error feedback control law in a complementary manner. Theoretical analysis indicates that the ILC-based traffic signal control methods can guarantee the asymptotic learning convergence, despite the presence of modeling uncertainties and exogenous disturbances. Finally, the impacts of the ILC-based signal control strategies on the network macroscopic fundamental diagram (MFD) are examined. The results show that the proposed ILC-based control strategies can homogenously distribute the network accumulation by controlling the vehicle numbers in each link to the desired levels under different traffic demands, which can result in the network with high capacity and mobility.

Support patient search on pathology reports with interactive online learning based data extraction.

PubMed

Zheng, Shuai; Lu, James J; Appin, Christina; Brat, Daniel; Wang, Fusheng

2015-01-01

Structural reporting enables semantic understanding and prompt retrieval of clinical findings about patients. While synoptic pathology reporting provides templates for data entries, information in pathology reports remains primarily in narrative free text form. Extracting data of interest from narrative pathology reports could significantly improve the representation of the information and enable complex structured queries. However, manual extraction is tedious and error-prone, and automated tools are often constructed with a fixed training dataset and not easily adaptable. Our goal is to extract data from pathology reports to support advanced patient search with a highly adaptable semi-automated data extraction system, which can adjust and self-improve by learning from a user's interaction with minimal human effort. We have developed an online machine learning based information extraction system called IDEAL-X. With its graphical user interface, the system's data extraction engine automatically annotates values for users to review upon loading each report text. The system analyzes users' corrections regarding these annotations with online machine learning, and incrementally enhances and refines the learning model as reports are processed. The system also takes advantage of customized controlled vocabularies, which can be adaptively refined during the online learning process to further assist the data extraction. As the accuracy of automatic annotation improves overtime, the effort of human annotation is gradually reduced. After all reports are processed, a built-in query engine can be applied to conveniently define queries based on extracted structured data. We have evaluated the system with a dataset of anatomic pathology reports from 50 patients. Extracted data elements include demographical data, diagnosis, genetic marker, and procedure. The system achieves F-1 scores of around 95% for the majority of tests. Extracting data from pathology reports could enable more accurate knowledge to support biomedical research and clinical diagnosis. IDEAL-X provides a bridge that takes advantage of online machine learning based data extraction and the knowledge from human's feedback. By combining iterative online learning and adaptive controlled vocabularies, IDEAL-X can deliver highly adaptive and accurate data extraction to support patient search.

Structured Kernel Subspace Learning for Autonomous Robot Navigation.

PubMed

Kim, Eunwoo; Choi, Sungjoon; Oh, Songhwai

2018-02-14

This paper considers two important problems for autonomous robot navigation in a dynamic environment, where the goal is to predict pedestrian motion and control a robot with the prediction for safe navigation. While there are several methods for predicting the motion of a pedestrian and controlling a robot to avoid incoming pedestrians, it is still difficult to safely navigate in a dynamic environment due to challenges, such as the varying quality and complexity of training data with unwanted noises. This paper addresses these challenges simultaneously by proposing a robust kernel subspace learning algorithm based on the recent advances in nuclear-norm and l 1 -norm minimization. We model the motion of a pedestrian and the robot controller using Gaussian processes. The proposed method efficiently approximates a kernel matrix used in Gaussian process regression by learning low-rank structured matrix (with symmetric positive semi-definiteness) to find an orthogonal basis, which eliminates the effects of erroneous and inconsistent data. Based on structured kernel subspace learning, we propose a robust motion model and motion controller for safe navigation in dynamic environments. We evaluate the proposed robust kernel learning in various tasks, including regression, motion prediction, and motion control problems, and demonstrate that the proposed learning-based systems are robust against outliers and outperform existing regression and navigation methods.

A Learning Support System Regarding Motion Trigger for Repetitive Motion Having an Operating Instrument

ERIC Educational Resources Information Center

Toyooka, Hiroshi; Matsuura, Kenji; Gotoda, Naka

2016-01-01

In the learning support for repetitive motions having an operating instrument, it is necessary for learners to control not only their own body motions but also an instrument corresponding to the body. This study focuses on the repetitive motion learning using single operation instrument without the movement in space; i.e. jump-rope and hula-hoop.…

Orbiter Water Dump Nozzles Redesign Lessons Learned

NASA Technical Reports Server (NTRS)

Rotter, Hank

2017-01-01

Hank Rotter, NASA Technical Fellow for Environmental Control and Life Support System, will provide the causes and lessons learned for the two Space Shuttle Orbiter water dump icicles that formed on the side of the Orbiter. He will present the root causes and the criticality of these icicles, along with the redesign of the water dump nozzles and lessons learned during the redesign phase.

Spatial Discrimination Reversal Learning in Weanling Rats Is Impaired by Striatal Administration of an NMDA-Receptor Antagonist

ERIC Educational Resources Information Center

Watson, Deborah J.; Stanton, Mark E.

2009-01-01

The striatum plays a major role in both motor control and learning and memory, including executive function and "behavioral flexibility." Lesion, temporary inactivation, and infusion of an N-methyl-d-aspartate (NMDA)-receptor antagonist into the dorsomedial striatum (dmSTR) impair reversal learning in adult rats. Systemic administration of MK-801…

An Evaluation of Pedagogical Tutorial Tactics for a Natural Language Tutoring System: A Reinforcement Learning Approach

ERIC Educational Resources Information Center

Chi, Min; VanLehn, Kurt; Litman, Diane; Jordan, Pamela

2011-01-01

Pedagogical strategies are policies for a tutor to decide the next action when there are multiple actions available. When the content is controlled to be the same across experimental conditions, there has been little evidence that tutorial decisions have an impact on students' learning. In this paper, we applied Reinforcement Learning (RL) to…

Privacy-Driven Design of Learning Analytics Applications: Exploring the Design Space of Solutions for Data Sharing and Interoperability

ERIC Educational Resources Information Center

Hoel, Tore; Chen, Weiqin

2016-01-01

Studies have shown that issues of privacy, control of data, and trust are essential to implementation of learning analytics systems. If these issues are not addressed appropriately, systems will tend to collapse due to a legitimacy crisis, or they will not be implemented in the first place due to resistance from learners, their parents, or their…

Climate Conditioning for the Learning Environment.

ERIC Educational Resources Information Center

Perkins and Will, Architects, Chicago, IL.

Discusses heating, cooling, and ventilation for the classroom in relationship to students' learning abilities. It is designed to assist school boards, administrators, architects and engineers in understanding the beneficial effects of total climate control, and in evaluating the climate conditioning systems available for schools. Discussion…

Designing the Undesignable: Social Software and Control

ERIC Educational Resources Information Center

Dron, Jon

2007-01-01

Social software, such as blogs, wikis, tagging systems and collaborative filters, treats the group as a first-class object within the system. Drawing from theories of transactional distance and control, this paper proposes a model of e-learning that extends traditional concepts of learner-teacher-content interactions to include these emergent…

Environmental Control System Installer/Servicer (Residential Air Conditioning Mechanic). V-TECS Guide.

ERIC Educational Resources Information Center

Meyer, Calvin F.; Benson, Robert T.

This guide provides job relevant tasks, performance objectives, performance guides, resources, learning activitites, evaluation standards, and achievement testing in the occupation of environmental control system installer/servicer (residential air conditioning mechanic). It is designed to be used with any chosen teaching method. The course…

Privacy Impact Assessment for the Confidential Business Information Records Access System for the Toxic Control Substances Act

EPA Pesticide Factsheets

This system collects submission data from the Toxic Substances Control Act (TSCA) and contact information for EPA contractors and employees who are CBI cleared. Learn how this data is collected, how it will be used, and the purpose of data collection.

Systemic Synthesis Questions [SSynQs] as Tools to Help Students to Build Their Cognitive Structures in a Systemic Manner

NASA Astrophysics Data System (ADS)

Hrin, Tamara N.; Fahmy, Ameen F. M.; Segedinac, Mirjana D.; Milenković, Dušica D.

2016-08-01

Many studies dedicated to the teaching and learning of organic chemistry courses have emphasized that high school students have shown significant difficulties in mastering the concepts of this discipline. Therefore, the aim of our study was to help students to overcome these difficulties by applying systemic synthesis questions, [SSynQs], as the instructional method in our intervention. This work shows that students from the group exposed to the new teaching method achieved higher scores on final testing than students from the control group, who were taught by the traditional method, when students' achievements in conventional, linear questions [LQs] and in [SSynQs] were studied. These results were followed by observation of lower levels of mental effort by students from the intervention group, and higher levels of mental effort in the control group, invested during solving both types of questions. This correlation between achievement and mental effort resulted in high instructional efficiency for the applied method in the intervention group, [SSynQs], and low instructional efficiency for the traditional teaching and learning method applied in the control group. A systemic triangular relation between achievement, mental effort, and instructional efficiency, established by each group and gender, emphasized that the application of [SSynQs] was more suited to female students than for male students because of [SSynQs] characteristics as teaching and learning tools and because of learning style and ability differences between genders.

Peripheral Inflammation Undermines the Plasticity of the Isolated Spinal Cord

PubMed Central

Huie, John R.; Grau, James W.

2009-01-01

Peripheral capsaicin treatment induces molecular changes that sensitize the responses of nociceptive neurons in the spinal dorsal horn. The current studies demonstrate that capsaicin also undermines the adaptive plasticity of the spinal cord, rendering the system incapable of learning a simple instrumental task. In these studies, male rats are transected at the second thoracic vertebra and are tested 24 to 48 hours later. During testing, subjects receive shock to one hindleg when it is extended (controllable stimulation). Rats quickly learn to maintain the leg in a flexed position. Rats that have been injected with capsaicin (1% or 3%) in the hindpaw fail to learn, even when tested on the leg contralateral to the injection. This learning deficit lasts at least 24 hours. Interestingly, training with controllable electrical stimulation prior to capsaicin administration protects the spinal cord against the maladaptive effects. Rats pretrained with controllable stimulation do not display a learning deficit or tactile allodynia. Moreover, controllable stimulation, combined with naltrexone, reverses the capsaicin-induced deficit. These data suggest that peripheral inflammation, accompanying spinal cord injuries, might have an adverse effect on recovery. PMID:18298266

Data-Driven Based Asynchronous Motor Control for Printing Servo Systems

NASA Astrophysics Data System (ADS)

Bian, Min; Guo, Qingyun

Modern digital printing equipment aims to the environmental-friendly industry with high dynamic performances and control precision and low vibration and abrasion. High performance motion control system of printing servo systems was required. Control system of asynchronous motor based on data acquisition was proposed. Iterative learning control (ILC) algorithm was studied. PID control was widely used in the motion control. However, it was sensitive to the disturbances and model parameters variation. The ILC applied the history error data and present control signals to approximate the control signal directly in order to fully track the expect trajectory without the system models and structures. The motor control algorithm based on the ILC and PID was constructed and simulation results were given. The results show that data-driven control method is effective dealing with bounded disturbances for the motion control of printing servo systems.

Cholinergic control of attention to cues guiding established performance versus learning: theoretical comment on Maddux, Kerfoot, Chatterjee, and Holland (2007).

PubMed

Sarter, Martin

2007-02-01

Previous views on the cognitive functions of the basal forebrain cholinergic system often suggested that this neuromodulator system influences fundamental attentional processes but not learning. The results from an elegant series of studies by J. M. Maddux, E. C. Kerfoot, S. Chatterjee, and P. Holland reveal the intricate relationships between the levels of attentional processing of stimuli and the rate of learning about such stimuli. Moreover, their results indicate a double dissociation between the role of prefrontal and posterior parietal cholinergic inputs, respectively, in attentional performance and the learning rate of stimuli that command different levels of attentional processing. Separate yet interacting components of the cortical cholinergic input system modulate the attentional processing of cues that guide well-practiced performance or that serve as conditioned stimuli during learning. Copyright (c) 2007 APA, all rights reserved.

Lessons Learned from Ares I Upper Stage Structures and Thermal Design

NASA Technical Reports Server (NTRS)

Ahmed, Rafiq

2012-01-01

The Ares 1 Upper Stage was part of the vehicle intended to succeed the Space Shuttle as the United States manned spaceflight vehicle. Although the Upper Stage project was cancelled, there were many lessons learned that are applicable to future vehicle design. Lessons learned that are briefly detailed in this Technical Memorandum are for specific technical areas such as tank design, common bulkhead design, thrust oscillation, control of flight and slosh loads, purge and hazardous gas system. In addition, lessons learned from a systems engineering and vehicle integration perspective are also included, such as computer aided design and engineering, scheduling, and data management. The need for detailed systems engineering in the early stages of a project is emphasized throughout this report. The intent is that future projects will be able to apply these lessons learned to keep costs down, schedules brief, and deliver products that perform to the expectations of their customers.

Control approaches for intelligent material systems -- What can we learn from nature?

DOE Office of Scientific and Technical Information (OSTI.GOV)

Robertshaw, H.H.

1994-12-31

Three natural systems (human thermoregulation, enzyme-catalyzed biochemical reactions, and rivers) are examined with the intent of finding commonalties in control among these systems which may offer inspiration or guidance to the task of controlling the behavior of Intelligent Material Systems. It is observed that these natural systems act in ways not seen in technological control systems. The observations of a lack of (feedback) control, the predominance of regulation, the extremely local nature of the apparent goals, the storage of information in form (in structure), and non-numerical processing, produce a strong impression of coupled open-loop processes amidst seeming chaos almost passivelymore » producing what the author calls natural system control.« less

Verification hybrid control of a wheeled mobile robot and manipulator

NASA Astrophysics Data System (ADS)

Muszynska, Magdalena; Burghardt, Andrzej; Kurc, Krzysztof; Szybicki, Dariusz

2016-04-01

In this article, innovative approaches to realization of the wheeled mobile robots and manipulator tracking are presented. Conceptions include application of the neural-fuzzy systems to compensation of the controlled system's nonlinearities in the tracking control task. Proposed control algorithms work on-line, contain structure, that adapt to the changeable work conditions of the controlled systems, and do not require the preliminary learning. The algorithm was verification on the real object which was a Scorbot - ER 4pc robotic manipulator and a Pioneer - 2DX mobile robot.

Choline acetyltransferase in the hippocampus is associated with learning strategy preference in adult male rats.

PubMed

Hawley, Wayne R; Witty, Christine F; Daniel, Jill M; Dohanich, Gary P

2015-08-01

One principle of the multiple memory systems hypothesis posits that the hippocampus-based and striatum-based memory systems compete for control over learning. Consistent with this notion, previous research indicates that the cholinergic system of the hippocampus plays a role in modulating the preference for a hippocampus-based place learning strategy over a striatum-based stimulus--response learning strategy. Interestingly, in the hippocampus, greater activity and higher protein levels of choline acetyltransferase (ChAT), the enzyme that synthesizes acetylcholine, are associated with better performance on hippocampus-based learning and memory tasks. With this in mind, the primary aim of the current study was to determine if higher levels of ChAT and the high-affinity choline uptake transporter (CHT) in the hippocampus were associated with a preference for a hippocampus-based place learning strategy on a task that also could be solved by relying on a striatum-based stimulus--response learning strategy. Results confirmed that levels of ChAT in the dorsal region of the hippocampus were associated with a preference for a place learning strategy on a water maze task that could also be solved by adopting a stimulus-response learning strategy. Consistent with previous studies, the current results support the hypothesis that the cholinergic system of the hippocampus plays a role in balancing competition between memory systems that modulate learning strategy preference. Copyright © 2015 Elsevier B.V. All rights reserved.

A Fully Automated Drosophila Olfactory Classical Conditioning and Testing System for Behavioral Learning and Memory Assessment

PubMed Central

Jiang, Hui; Hanna, Eriny; Gatto, Cheryl L.; Page, Terry L.; Bhuva, Bharat; Broadie, Kendal

2016-01-01

Background Aversive olfactory classical conditioning has been the standard method to assess Drosophila learning and memory behavior for decades, yet training and testing are conducted manually under exceedingly labor-intensive conditions. To overcome this severe limitation, a fully automated, inexpensive system has been developed, which allows accurate and efficient Pavlovian associative learning/memory analyses for high-throughput pharmacological and genetic studies. New Method The automated system employs a linear actuator coupled to an odorant T-maze with airflow-mediated transfer of animals between training and testing stages. Odorant, airflow and electrical shock delivery are automatically administered and monitored during training trials. Control software allows operator-input variables to define parameters of Drosophila learning, short-term memory and long-term memory assays. Results The approach allows accurate learning/memory determinations with operational fail-safes. Automated learning indices (immediately post-training) and memory indices (after 24 hours) are comparable to traditional manual experiments, while minimizing experimenter involvement. Comparison with Existing Methods The automated system provides vast improvements over labor-intensive manual approaches with no experimenter involvement required during either training or testing phases. It provides quality control tracking of airflow rates, odorant delivery and electrical shock treatments, and an expanded platform for high-throughput studies of combinational drug tests and genetic screens. The design uses inexpensive hardware and software for a total cost of ~$500US, making it affordable to a wide range of investigators. Conclusions This study demonstrates the design, construction and testing of a fully automated Drosophila olfactory classical association apparatus to provide low-labor, high-fidelity, quality-monitored, high-throughput and inexpensive learning and memory behavioral assays. PMID:26703418

A fully automated Drosophila olfactory classical conditioning and testing system for behavioral learning and memory assessment.

PubMed

Jiang, Hui; Hanna, Eriny; Gatto, Cheryl L; Page, Terry L; Bhuva, Bharat; Broadie, Kendal

2016-03-01

Aversive olfactory classical conditioning has been the standard method to assess Drosophila learning and memory behavior for decades, yet training and testing are conducted manually under exceedingly labor-intensive conditions. To overcome this severe limitation, a fully automated, inexpensive system has been developed, which allows accurate and efficient Pavlovian associative learning/memory analyses for high-throughput pharmacological and genetic studies. The automated system employs a linear actuator coupled to an odorant T-maze with airflow-mediated transfer of animals between training and testing stages. Odorant, airflow and electrical shock delivery are automatically administered and monitored during training trials. Control software allows operator-input variables to define parameters of Drosophila learning, short-term memory and long-term memory assays. The approach allows accurate learning/memory determinations with operational fail-safes. Automated learning indices (immediately post-training) and memory indices (after 24h) are comparable to traditional manual experiments, while minimizing experimenter involvement. The automated system provides vast improvements over labor-intensive manual approaches with no experimenter involvement required during either training or testing phases. It provides quality control tracking of airflow rates, odorant delivery and electrical shock treatments, and an expanded platform for high-throughput studies of combinational drug tests and genetic screens. The design uses inexpensive hardware and software for a total cost of ∼$500US, making it affordable to a wide range of investigators. This study demonstrates the design, construction and testing of a fully automated Drosophila olfactory classical association apparatus to provide low-labor, high-fidelity, quality-monitored, high-throughput and inexpensive learning and memory behavioral assays. Copyright © 2015 Elsevier B.V. All rights reserved.

Early Foundations for Mathematics Learning and Their Relations to Learning Disabilities.

PubMed

Geary, David C

2013-02-01

Children's quantitative competencies upon entry into school can have lifelong consequences. Children who start behind generally stay behind, and mathematical skills at school completion influence employment prospects and wages in adulthood. I review the current debate over whether early quantitative learning is supported by (a) an inherent system for representing approximate magnitudes, (b) an attentional-control system that enables explicit processing of quantitative symbols, such as Arabic numerals, or (c) the logical problem-solving abilities that facilitate learning of the relations among numerals. Studies of children with mathematical learning disabilities and difficulties have suggested that each of these competencies may be involved, but to different degrees and at different points in the learning process. Clarifying how and when these competencies facilitate early quantitative learning and developing interventions to address their impact on children have the potential to yield substantial benefits for individuals and for society.

Instructional control of reinforcement learning: A behavioral and neurocomputational investigation

PubMed Central

Doll, Bradley B.; Jacobs, W. Jake; Sanfey, Alan G.; Frank, Michael J.

2011-01-01

Humans learn how to behave directly through environmental experience and indirectly through rules and instructions. Behavior analytic research has shown that instructions can control behavior, even when such behavior leads to sub-optimal outcomes (Hayes, S. (Ed.). 1989. Rule-governed behavior: cognition, contingencies, and instructional control. Plenum Press.). Here we examine the control of behavior through instructions in a reinforcement learning task known to depend on striatal dopaminergic function. Participants selected between probabilistically reinforced stimuli, and were (incorrectly) told that a specific stimulus had the highest (or lowest) reinforcement probability. Despite experience to the contrary, instructions drove choice behavior. We present neural network simulations that capture the interactions between instruction-driven and reinforcement-driven behavior via two potential neural circuits: one in which the striatum is inaccurately trained by instruction representations coming from prefrontal cortex/hippocampus (PFC/HC), and another in which the striatum learns the environmentally based reinforcement contingencies, but is “overridden” at decision output. Both models capture the core behavioral phenomena but, because they differ fundamentally on what is learned, make distinct predictions for subsequent behavioral and neuroimaging experiments. Finally, we attempt to distinguish between the proposed computational mechanisms governing instructed behavior by fitting a series of abstract “Q-learning” and Bayesian models to subject data. The best-fitting model supports one of the neural models, suggesting the existence of a “confirmation bias” in which the PFC/HC system trains the reinforcement system by amplifying outcomes that are consistent with instructions while diminishing inconsistent outcomes. PMID:19595993

Systems autonomy

NASA Technical Reports Server (NTRS)

Lum, Henry, Jr.

1988-01-01

Information on systems autonomy is given in viewgraph form. Information is given on space systems integration, intelligent autonomous systems, automated systems for in-flight mission operations, the Systems Autonomy Demonstration Project on the Space Station Thermal Control System, the architecture of an autonomous intelligent system, artificial intelligence research issues, machine learning, and real-time image processing.

NASA Workshop on Hybrid (Mixed-Actuator) Spacecraft Attitude Control

NASA Technical Reports Server (NTRS)

Dennehy, Cornelius J.; Kunz, Nans

2014-01-01

At the request of the Science Mission Directorate Chief Engineer, the NASA Technical Fellow for Guidance, Navigation & Control assembled and facilitated a workshop on Spacecraft Hybrid Attitude Control. This multi-Center, academic, and industry workshop, sponsored by the NASA Engineering and Safety Center (NESC), was held in April 2013 to unite nationwide experts to present and discuss the various innovative solutions, techniques, and lessons learned regarding the development and implementation of the various hybrid attitude control system solutions investigated or implemented. This report attempts to document these key lessons learned with the 16 findings and 9 NESC recommendations.

Application of a repetitive process setting to design of monotonically convergent iterative learning control

NASA Astrophysics Data System (ADS)

Boski, Marcin; Paszke, Wojciech

2015-11-01

This paper deals with the problem of designing an iterative learning control algorithm for discrete linear systems using repetitive process stability theory. The resulting design produces a stabilizing output feedback controller in the time domain and a feedforward controller that guarantees monotonic convergence in the trial-to-trial domain. The results are also extended to limited frequency range design specification. New design procedure is introduced in terms of linear matrix inequality (LMI) representations, which guarantee the prescribed performances of ILC scheme. A simulation example is given to illustrate the theoretical developments.

Kernel-based least squares policy iteration for reinforcement learning.

PubMed

Xu, Xin; Hu, Dewen; Lu, Xicheng

2007-07-01

In this paper, we present a kernel-based least squares policy iteration (KLSPI) algorithm for reinforcement learning (RL) in large or continuous state spaces, which can be used to realize adaptive feedback control of uncertain dynamic systems. By using KLSPI, near-optimal control policies can be obtained without much a priori knowledge on dynamic models of control plants. In KLSPI, Mercer kernels are used in the policy evaluation of a policy iteration process, where a new kernel-based least squares temporal-difference algorithm called KLSTD-Q is proposed for efficient policy evaluation. To keep the sparsity and improve the generalization ability of KLSTD-Q solutions, a kernel sparsification procedure based on approximate linear dependency (ALD) is performed. Compared to the previous works on approximate RL methods, KLSPI makes two progresses to eliminate the main difficulties of existing results. One is the better convergence and (near) optimality guarantee by using the KLSTD-Q algorithm for policy evaluation with high precision. The other is the automatic feature selection using the ALD-based kernel sparsification. Therefore, the KLSPI algorithm provides a general RL method with generalization performance and convergence guarantee for large-scale Markov decision problems (MDPs). Experimental results on a typical RL task for a stochastic chain problem demonstrate that KLSPI can consistently achieve better learning efficiency and policy quality than the previous least squares policy iteration (LSPI) algorithm. Furthermore, the KLSPI method was also evaluated on two nonlinear feedback control problems, including a ship heading control problem and the swing up control of a double-link underactuated pendulum called acrobot. Simulation results illustrate that the proposed method can optimize controller performance using little a priori information of uncertain dynamic systems. It is also demonstrated that KLSPI can be applied to online learning control by incorporating an initial controller to ensure online performance.

System Identification for Nonlinear Control Using Neural Networks

NASA Technical Reports Server (NTRS)

Stengel, Robert F.; Linse, Dennis J.

1990-01-01

An approach to incorporating artificial neural networks in nonlinear, adaptive control systems is described. The controller contains three principal elements: a nonlinear inverse dynamic control law whose coefficients depend on a comprehensive model of the plant, a neural network that models system dynamics, and a state estimator whose outputs drive the control law and train the neural network. Attention is focused on the system identification task, which combines an extended Kalman filter with generalized spline function approximation. Continual learning is possible during normal operation, without taking the system off line for specialized training. Nonlinear inverse dynamic control requires smooth derivatives as well as function estimates, imposing stringent goals on the approximating technique.

Bio-inspired spiking neural network for nonlinear systems control.

PubMed

Pérez, Javier; Cabrera, Juan A; Castillo, Juan J; Velasco, Juan M

2018-08-01

Spiking neural networks (SNN) are the third generation of artificial neural networks. SNN are the closest approximation to biological neural networks. SNNs make use of temporal spike trains to command inputs and outputs, allowing a faster and more complex computation. As demonstrated by biological organisms, they are a potentially good approach to designing controllers for highly nonlinear dynamic systems in which the performance of controllers developed by conventional techniques is not satisfactory or difficult to implement. SNN-based controllers exploit their ability for online learning and self-adaptation to evolve when transferred from simulations to the real world. SNN's inherent binary and temporary way of information codification facilitates their hardware implementation compared to analog neurons. Biological neural networks often require a lower number of neurons compared to other controllers based on artificial neural networks. In this work, these neuronal systems are imitated to perform the control of non-linear dynamic systems. For this purpose, a control structure based on spiking neural networks has been designed. Particular attention has been paid to optimizing the structure and size of the neural network. The proposed structure is able to control dynamic systems with a reduced number of neurons and connections. A supervised learning process using evolutionary algorithms has been carried out to perform controller training. The efficiency of the proposed network has been verified in two examples of dynamic systems control. Simulations show that the proposed control based on SNN exhibits superior performance compared to other approaches based on Neural Networks and SNNs. Copyright © 2018 Elsevier Ltd. All rights reserved.

The Representation of Motor (Inter)action, States of Action, and Learning: Three Perspectives on Motor Learning by Way of Imagery and Execution

PubMed Central

Frank, Cornelia; Schack, Thomas

2017-01-01

Learning in intelligent systems is a result of direct and indirect interaction with the environment. While humans can learn by way of different states of (inter)action such as the execution or the imagery of an action, their unique potential to induce brain- and mind-related changes in the motor action system is still being debated. The systematic repetition of different states of action (e.g., physical and/or mental practice) and their contribution to the learning of complex motor actions has traditionally been approached by way of performance improvements. More recently, approaches highlighting the role of action representation in the learning of complex motor actions have evolved and may provide additional insight into the learning process. In the present perspective paper, we build on brain-related findings and sketch recent research on learning by way of imagery and execution from a hierarchical, perceptual-cognitive approach to motor control and learning. These findings provide insights into the learning of intelligent systems from a perceptual-cognitive, representation-based perspective and as such add to our current understanding of action representation in memory and its changes with practice. Future research should build bridges between approaches in order to more thoroughly understand functional changes throughout the learning process and to facilitate motor learning, which may have particular importance for cognitive systems research in robotics, rehabilitation, and sports. PMID:28588510

WNN 92; Proceedings of the 3rd Workshop on Neural Networks: Academic/Industrial/NASA/Defense, Auburn Univ., AL, Feb. 10-12, 1992 and South Shore Harbour, TX, Nov. 4-6, 1992

NASA Technical Reports Server (NTRS)

Padgett, Mary L. (Editor)

1993-01-01

The present conference discusses such neural networks (NN) related topics as their current development status, NN architectures, NN learning rules, NN optimization methods, NN temporal models, NN control methods, NN pattern recognition systems and applications, biological and biomedical applications of NNs, VLSI design techniques for NNs, NN systems simulation, fuzzy logic, and genetic algorithms. Attention is given to missileborne integrated NNs, adaptive-mixture NNs, implementable learning rules, an NN simulator for travelling salesman problem solutions, similarity-based forecasting, NN control of hypersonic aircraft takeoff, NN control of the Space Shuttle Arm, an adaptive NN robot manipulator controller, a synthetic approach to digital filtering, NNs for speech analysis, adaptive spline networks, an anticipatory fuzzy logic controller, and encoding operations for fuzzy associative memories.

Deep learning and model predictive control for self-tuning mode-locked lasers

NASA Astrophysics Data System (ADS)

Baumeister, Thomas; Brunton, Steven L.; Nathan Kutz, J.

2018-03-01

Self-tuning optical systems are of growing importance in technological applications such as mode-locked fiber lasers. Such self-tuning paradigms require {\\em intelligent} algorithms capable of inferring approximate models of the underlying physics and discovering appropriate control laws in order to maintain robust performance for a given objective. In this work, we demonstrate the first integration of a {\\em deep learning} (DL) architecture with {\\em model predictive control} (MPC) in order to self-tune a mode-locked fiber laser. Not only can our DL-MPC algorithmic architecture approximate the unknown fiber birefringence, it also builds a dynamical model of the laser and appropriate control law for maintaining robust, high-energy pulses despite a stochastically drifting birefringence. We demonstrate the effectiveness of this method on a fiber laser which is mode-locked by nonlinear polarization rotation. The method advocated can be broadly applied to a variety of optical systems that require robust controllers.

Model-free learning on robot kinematic chains using a nested multi-agent topology

NASA Astrophysics Data System (ADS)

Karigiannis, John N.; Tzafestas, Costas S.

2016-11-01

This paper proposes a model-free learning scheme for the developmental acquisition of robot kinematic control and dexterous manipulation skills. The approach is based on a nested-hierarchical multi-agent architecture that intuitively encapsulates the topology of robot kinematic chains, where the activity of each independent degree-of-freedom (DOF) is finally mapped onto a distinct agent. Each one of those agents progressively evolves a local kinematic control strategy in a game-theoretic sense, that is, based on a partial (local) view of the whole system topology, which is incrementally updated through a recursive communication process according to the nested-hierarchical topology. Learning is thus approached not through demonstration and training but through an autonomous self-exploration process. A fuzzy reinforcement learning scheme is employed within each agent to enable efficient exploration in a continuous state-action domain. This paper constitutes in fact a proof of concept, demonstrating that global dexterous manipulation skills can indeed evolve through such a distributed iterative learning of local agent sensorimotor mappings. The main motivation behind the development of such an incremental multi-agent topology is to enhance system modularity, to facilitate extensibility to more complex problem domains and to improve robustness with respect to structural variations including unpredictable internal failures. These attributes of the proposed system are assessed in this paper through numerical experiments in different robot manipulation task scenarios, involving both single and multi-robot kinematic chains. The generalisation capacity of the learning scheme is experimentally assessed and robustness properties of the multi-agent system are also evaluated with respect to unpredictable variations in the kinematic topology. Furthermore, these numerical experiments demonstrate the scalability properties of the proposed nested-hierarchical architecture, where new agents can be recursively added in the hierarchy to encapsulate individual active DOFs. The results presented in this paper demonstrate the feasibility of such a distributed multi-agent control framework, showing that the solutions which emerge are plausible and near-optimal. Numerical efficiency and computational cost issues are also discussed.

Organisational reporting and learning systems: Innovating inside and outside of the box.

PubMed

Sujan, Mark; Furniss, Dominic

2015-01-01

Reporting and learning systems are key organisational tools for the management and prevention of clinical risk. However, current approaches, such as incident reporting, are struggling to meet expectations of turning health systems like the UK National Health Service (NHS) into learning organisations. This article aims to open up debate on the potential for novel reporting and learning systems in healthcare, by reflecting on experiences from two recent projects: Proactive Risk Monitoring in Healthcare (PRIMO) and Errordiary in Healthcare. These two approaches demonstrate how paying attention to ordinary, everyday clinical work can derive useful learning and active discussion about clinical risk. We argue that innovations in reporting and learning systems might come from both inside and outside of the box. 'Inside' being along traditional paths of controlled organisational innovation. 'Outside' in the sense that inspiration comes outside of the healthcare domain, or more extremely, outside official channels through external websites and social media (e.g. patient forums, public review sites, whistleblower blogs and Twitter streams). Reporting routes that bypass official channels could empower staff and patient activism, and turn out to be a driver to challenge organisational processes, assumptions and priorities where the organisation is failing and has become unresponsive.

Sensory motor remapping of space in human–machine interfaces

PubMed Central

Mussa-Ivaldi, Ferdinando A.; Casadio, Maura; Danziger, Zachary C.; Mosier, Kristine M.; Scheidt, Robert A.

2012-01-01

Studies of adaptation to patterns of deterministic forces have revealed the ability of the motor control system to form and use predictive representations of the environment. These studies have also pointed out that adaptation to novel dynamics is aimed at preserving the trajectories of a controlled endpoint, either the hand of a subject or a transported object. We review some of these experiments and present more recent studies aimed at understanding how the motor system forms representations of the physical space in which actions take place. An extensive line of investigations in visual information processing has dealt with the issue of how the Euclidean properties of space are recovered from visual signals that do not appear to possess these properties. The same question is addressed here in the context of motor behavior and motor learning by observing how people remap hand gestures and body motions that control the state of an external device. We present some theoretical considerations and experimental evidence about the ability of the nervous system to create novel patterns of coordination that are consistent with the representation of extrapersonal space. We also discuss the perspective of endowing human–machine interfaces with learning algorithms that, combined with human learning, may facilitate the control of powered wheelchairs and other assistive devices. PMID:21741543

Learning Based Bidding Strategy for HVAC Systems in Double Auction Retail Energy Markets

DOE Office of Scientific and Technical Information (OSTI.GOV)

Sun, Yannan; Somani, Abhishek; Carroll, Thomas E.

In this paper, a bidding strategy is proposed using reinforcement learning for HVAC systems in a double auction market. The bidding strategy does not require a specific model-based representation of behavior, i.e., a functional form to translate indoor house temperatures into bid prices. The results from reinforcement learning based approach are compared with the HVAC bidding approach used in the AEP gridSMART® smart grid demonstration project and it is shown that the model-free (learning based) approach tracks well the results from the model-based behavior. Successful use of model-free approaches to represent device-level economic behavior may help develop similar approaches tomore » represent behavior of more complex devices or groups of diverse devices, such as in a building. Distributed control requires an understanding of decision making processes of intelligent agents so that appropriate mechanisms may be developed to control and coordinate their responses, and model-free approaches to represent behavior will be extremely useful in that quest.« less

Using Fuzzy Logic for Performance Evaluation in Reinforcement Learning

NASA Technical Reports Server (NTRS)

Berenji, Hamid R.; Khedkar, Pratap S.

1992-01-01

Current reinforcement learning algorithms require long training periods which generally limit their applicability to small size problems. A new architecture is described which uses fuzzy rules to initialize its two neural networks: a neural network for performance evaluation and another for action selection. This architecture is applied to control of dynamic systems and it is demonstrated that it is possible to start with an approximate prior knowledge and learn to refine it through experiments using reinforcement learning.

Adaptive optimal control of unknown constrained-input systems using policy iteration and neural networks.

PubMed

Modares, Hamidreza; Lewis, Frank L; Naghibi-Sistani, Mohammad-Bagher

2013-10-01

This paper presents an online policy iteration (PI) algorithm to learn the continuous-time optimal control solution for unknown constrained-input systems. The proposed PI algorithm is implemented on an actor-critic structure where two neural networks (NNs) are tuned online and simultaneously to generate the optimal bounded control policy. The requirement of complete knowledge of the system dynamics is obviated by employing a novel NN identifier in conjunction with the actor and critic NNs. It is shown how the identifier weights estimation error affects the convergence of the critic NN. A novel learning rule is developed to guarantee that the identifier weights converge to small neighborhoods of their ideal values exponentially fast. To provide an easy-to-check persistence of excitation condition, the experience replay technique is used. That is, recorded past experiences are used simultaneously with current data for the adaptation of the identifier weights. Stability of the whole system consisting of the actor, critic, system state, and system identifier is guaranteed while all three networks undergo adaptation. Convergence to a near-optimal control law is also shown. The effectiveness of the proposed method is illustrated with a simulation example.

The efficacy of using human myoelectric signals to control the limbs of robots in space

NASA Technical Reports Server (NTRS)

Clark, Jane E.; Phillips, Sally J.

1988-01-01

This project was designed to investigate the usefulness of the myoelectric signal as a control in robotics applications. More specifically, the neural patterns associated with human arm and hand actions were studied to determine the efficacy of using these myoelectric signals to control the manipulator arm of a robot. The advantage of this approach to robotic control was the use of well-defined and well-practiced neural patterns already available to the system, as opposed to requiring the human operator to learn new tasks and establish new neural patterns in learning to control a joystick or mechanical coupling device.

The basal ganglia is necessary for learning spectral, but not temporal features of birdsong

PubMed Central

Ali, Farhan; Fantana, Antoniu L.; Burak, Yoram; Ölveczky, Bence P.

2013-01-01

Executing a motor skill requires the brain to control which muscles to activate at what times. How these aspects of control - motor implementation and timing - are acquired, and whether the learning processes underlying them differ, is not well understood. To address this we used a reinforcement learning paradigm to independently manipulate both spectral and temporal features of birdsong, a complex learned motor sequence, while recording and perturbing activity in underlying circuits. Our results uncovered a striking dissociation in how neural circuits underlie learning in the two domains. The basal ganglia was required for modifying spectral, but not temporal structure. This functional dissociation extended to the descending motor pathway, where recordings from a premotor cortex analogue nucleus reflected changes to temporal, but not spectral structure. Our results reveal a strategy in which the nervous system employs different and largely independent circuits to learn distinct aspects of a motor skill. PMID:24075977

Cognitive Control Predicts Use of Model-Based Reinforcement-Learning

PubMed Central

Otto, A. Ross; Skatova, Anya; Madlon-Kay, Seth; Daw, Nathaniel D.

2015-01-01

Accounts of decision-making and its neural substrates have long posited the operation of separate, competing valuation systems in the control of choice behavior. Recent theoretical and experimental work suggest that this classic distinction between behaviorally and neurally dissociable systems for habitual and goal-directed (or more generally, automatic and controlled) choice may arise from two computational strategies for reinforcement learning (RL), called model-free and model-based RL, but the cognitive or computational processes by which one system may dominate over the other in the control of behavior is a matter of ongoing investigation. To elucidate this question, we leverage the theoretical framework of cognitive control, demonstrating that individual differences in utilization of goal-related contextual information—in the service of overcoming habitual, stimulus-driven responses—in established cognitive control paradigms predict model-based behavior in a separate, sequential choice task. The behavioral correspondence between cognitive control and model-based RL compellingly suggests that a common set of processes may underpin the two behaviors. In particular, computational mechanisms originally proposed to underlie controlled behavior may be applicable to understanding the interactions between model-based and model-free choice behavior. PMID:25170791

Safe Exploration Algorithms for Reinforcement Learning Controllers.

PubMed

Mannucci, Tommaso; van Kampen, Erik-Jan; de Visser, Cornelis; Chu, Qiping

2018-04-01

Self-learning approaches, such as reinforcement learning, offer new possibilities for autonomous control of uncertain or time-varying systems. However, exploring an unknown environment under limited prediction capabilities is a challenge for a learning agent. If the environment is dangerous, free exploration can result in physical damage or in an otherwise unacceptable behavior. With respect to existing methods, the main contribution of this paper is the definition of a new approach that does not require global safety functions, nor specific formulations of the dynamics or of the environment, but relies on interval estimation of the dynamics of the agent during the exploration phase, assuming a limited capability of the agent to perceive the presence of incoming fatal states. Two algorithms are presented with this approach. The first is the Safety Handling Exploration with Risk Perception Algorithm (SHERPA), which provides safety by individuating temporary safety functions, called backups. SHERPA is shown in a simulated, simplified quadrotor task, for which dangerous states are avoided. The second algorithm, denominated OptiSHERPA, can safely handle more dynamically complex systems for which SHERPA is not sufficient through the use of safety metrics. An application of OptiSHERPA is simulated on an aircraft altitude control task.

An H(∞) control approach to robust learning of feedforward neural networks.

PubMed

Jing, Xingjian

2011-09-01

A novel H(∞) robust control approach is proposed in this study to deal with the learning problems of feedforward neural networks (FNNs). The analysis and design of a desired weight update law for the FNN is transformed into a robust controller design problem for a discrete dynamic system in terms of the estimation error. The drawbacks of some existing learning algorithms can therefore be revealed, especially for the case that the output data is fast changing with respect to the input or the output data is corrupted by noise. Based on this approach, the optimal learning parameters can be found by utilizing the linear matrix inequality (LMI) optimization techniques to achieve a predefined H(∞) "noise" attenuation level. Several existing BP-type algorithms are shown to be special cases of the new H(∞)-learning algorithm. Theoretical analysis and several examples are provided to show the advantages of the new method. Copyright © 2011 Elsevier Ltd. All rights reserved.

Development of an e-learning system for teaching endoscopists how to diagnose early gastric cancer: basic principles for improving early detection.

PubMed

Yao, Kenshi; Uedo, Noriya; Muto, Manabu; Ishikawa, Hideki

2017-03-01

We developed an internet e-learning system in order to improve the ability of endoscopists to diagnose gastric cancer at an early stage. The efficacy of this system at expanding knowledge and providing invaluable experience regarding the endoscopic detection of early gastric cancer was demonstrated through an international multicenter randomized controlled trial. However, the contents of the system have not yet been fully described in the literature. Accordingly, we herein introduce the contents and their principles, which comprise three main subjects: technique, knowledge, and experience. Since all the e-learning contents and principles are based on conventional white-light endoscopy alone, which is commonly available throughout the world, they should provide a good reference point for any endoscopist who wishes to devise learning materials and guidelines for improving their own clinical practice.

Enhanced Muscle Afferent Signals during Motor Learning in Humans.

PubMed

Dimitriou, Michael

2016-04-25

Much has been revealed concerning human motor learning at the behavioral level [1, 2], but less is known about changes in the involved neural circuits and signals. By examining muscle spindle responses during a classic visuomotor adaptation task [3-6] performed by fully alert humans, I found substantial modulation of sensory afferent signals as a function of adaptation state. Specifically, spindle control was independent of concurrent muscle activity but was specific to movement direction (representing muscle lengthening versus shortening) and to different stages of learning. Increased spindle afferent responses to muscle stretch occurring early during learning reflected individual error size and were negatively related to subsequent antagonist activity (i.e., 60-80 ms thereafter). Relative increases in tonic afferent output early during learning were predictive of the subjects' adaptation rate. I also found that independent spindle control during sensory realignment (the "washout" stage) induced afferent signal "linearization" with respect to muscle length (i.e., signals were more tuned to hand position). The results demonstrate for the first time that motor learning also involves independent and state-related modulation of sensory mechanoreceptor signals. The current findings suggest that adaptive motor performance also relies on the independent control of sensors, not just of muscles. I propose that the "γ" motor system innervating spindles acts to facilitate the acquisition and extraction of task-relevant information at the early stages of sensorimotor adaptation. This designates a more active and targeted role for the human proprioceptive system during motor learning. Copyright © 2016 Elsevier Ltd. All rights reserved.

Formative Evaluation of a Massively Multi-Player Persistent (MMP) Environment for Asymmetric Warfare Exercises

DTIC Science & Technology

2008-04-01

Initiative acknowledges the dearth of published research on Massively Multiplayer Online Games ((MMOGs), which are based on MMP technology) for...wanting help during the emergency, protesting insufficient aid being delivered in time, escalating to violence , including sniper attacks and attempted... multiplayer environment. The movement control systems were rated "moderately easy" to learn, and ease of movement after learning to use the controls was rated

Learning-based traffic signal control algorithms with neighborhood information sharing: An application for sustainable mobility

DOE Office of Scientific and Technical Information (OSTI.GOV)

Aziz, H. M. Abdul; Zhu, Feng; Ukkusuri, Satish V.

Here, this research applies R-Markov Average Reward Technique based reinforcement learning (RL) algorithm, namely RMART, for vehicular signal control problem leveraging information sharing among signal controllers in connected vehicle environment. We implemented the algorithm in a network of 18 signalized intersections and compare the performance of RMART with fixed, adaptive, and variants of the RL schemes. Results show significant improvement in system performance for RMART algorithm with information sharing over both traditional fixed signal timing plans and real time adaptive control schemes. Additionally, the comparison with reinforcement learning algorithms including Q learning and SARSA indicate that RMART performs better atmore » higher congestion levels. Further, a multi-reward structure is proposed that dynamically adjusts the reward function with varying congestion states at the intersection. Finally, the results from test networks show significant reduction in emissions (CO, CO 2, NO x, VOC, PM 10) when RL algorithms are implemented compared to fixed signal timings and adaptive schemes.« less

Development of a neuromorphic control system for a lightweight humanoid robot

NASA Astrophysics Data System (ADS)

Folgheraiter, Michele; Keldibek, Amina; Aubakir, Bauyrzhan; Salakchinov, Shyngys; Gini, Giuseppina; Mauro Franchi, Alessio; Bana, Matteo

2017-03-01

A neuromorphic control system for a lightweight middle size humanoid biped robot built using 3D printing techniques is proposed. The control architecture consists of different modules capable to learn and autonomously reproduce complex periodic trajectories. Each module is represented by a chaotic Recurrent Neural Network (RNN) with a core of dynamic neurons randomly and sparsely connected with fixed synapses. A set of read-out units with adaptable synapses realize a linear combination of the neurons output in order to reproduce the target signals. Different experiments were conducted to find out the optimal initialization for the RNN’s parameters. From simulation results, using normalized signals obtained from the robot model, it was proven that all the instances of the control module can learn and reproduce the target trajectories with an average RMS error of 1.63 and variance 0.74.

The Impact of Using Mobile-Supported Learning Management Systems in Teaching Web Design on the Academic Success of Students and Their Opinions on the Course

ERIC Educational Resources Information Center

Tezer, Murat; Çimsir, Burcu Turan

2018-01-01

This research aimed to examine the impact of using mobile-supported learning management systems (LMS) in teaching web design on the academic success of students and their opinion on the course; and it was conducted on 70 volunteer students (35 experimental, 35 control) enrolled at Giresun University, Technical Sciences Vocational School, Computer…

On-line integration of computer controlled diagnostic devices and medical information systems in undergraduate medical physics education for physicians.

PubMed

Hanus, Josef; Nosek, Tomas; Zahora, Jiri; Bezrouk, Ales; Masin, Vladimir

2013-01-01

We designed and evaluated an innovative computer-aided-learning environment based on the on-line integration of computer controlled medical diagnostic devices and a medical information system for use in the preclinical medical physics education of medical students. Our learning system simulates the actual clinical environment in a hospital or primary care unit. It uses a commercial medical information system for on-line storage and processing of clinical type data acquired during physics laboratory classes. Every student adopts two roles, the role of 'patient' and the role of 'physician'. As a 'physician' the student operates the medical devices to clinically assess 'patient' colleagues and records all results in an electronic 'patient' record. We also introduced an innovative approach to the use of supportive education materials, based on the methods of adaptive e-learning. A survey of student feedback is included and statistically evaluated. The results from the student feedback confirm the positive response of the latter to this novel implementation of medical physics and informatics in preclinical education. This approach not only significantly improves learning of medical physics and informatics skills but has the added advantage that it facilitates students' transition from preclinical to clinical subjects. Copyright © 2011 Associazione Italiana di Fisica Medica. Published by Elsevier Ltd. All rights reserved.

The relationship of neurogenesis and growth of brain regions to song learning.

PubMed

Kirn, John R

2010-10-01

Song learning, maintenance and production require coordinated activity across multiple auditory, sensory-motor, and neuromuscular structures. Telencephalic components of the sensory-motor circuitry are unique to avian species that engage in song learning. The song system shows protracted development that begins prior to hatching but continues well into adulthood. The staggered developmental timetable for construction of the song system provides clues of subsystems involved in specific stages of song learning and maintenance. Progressive events, including neurogenesis and song system growth, as well as regressive events such as apoptosis and synapse elimination, occur during periods of song learning and the transitions between variable and stereotyped song during both development and adulthood. There is clear evidence that gonadal steroids influence the development of song attributes and shape the underlying neural circuitry. Some aspects of song system development are influenced by sensory, motor and social experience, while other aspects of neural development appear to be experience-independent. Although there are species differences in the extent to which song learning continues into adulthood, growing evidence suggests that despite differences in learning trajectories, adult refinement of song motor control and song maintenance can require remarkable behavioral and neural flexibility reminiscent of sensory-motor learning. Copyright © 2009 Elsevier Inc. All rights reserved.

Metabolic learning and memory formation by the brain influence systemic metabolic homeostasis.

PubMed

Zhang, Yumin; Liu, Gang; Yan, Jingqi; Zhang, Yalin; Li, Bo; Cai, Dongsheng

2015-04-07

Metabolic homeostasis is regulated by the brain, but whether this regulation involves learning and memory of metabolic information remains unexplored. Here we use a calorie-based, taste-independent learning/memory paradigm to show that Drosophila form metabolic memories that help in balancing food choice with caloric intake; however, this metabolic learning or memory is lost under chronic high-calorie feeding. We show that loss of individual learning/memory-regulating genes causes a metabolic learning defect, leading to elevated trehalose and lipid levels. Importantly, this function of metabolic learning requires not only the mushroom body but also the hypothalamus-like pars intercerebralis, while NF-κB activation in the pars intercerebralis mimics chronic overnutrition in that it causes metabolic learning impairment and disorders. Finally, we evaluate this concept of metabolic learning/memory in mice, suggesting that the hypothalamus is involved in a form of nutritional learning and memory, which is critical for determining resistance or susceptibility to obesity. In conclusion, our data indicate that the brain, and potentially the hypothalamus, direct metabolic learning and the formation of memories, which contribute to the control of systemic metabolic homeostasis.

New supervised learning theory applied to cerebellar modeling for suppression of variability of saccade end points.

PubMed

Fujita, Masahiko

2013-06-01

A new supervised learning theory is proposed for a hierarchical neural network with a single hidden layer of threshold units, which can approximate any continuous transformation, and applied to a cerebellar function to suppress the end-point variability of saccades. In motor systems, feedback control can reduce noise effects if the noise is added in a pathway from a motor center to a peripheral effector; however, it cannot reduce noise effects if the noise is generated in the motor center itself: a new control scheme is necessary for such noise. The cerebellar cortex is well known as a supervised learning system, and a novel theory of cerebellar cortical function developed in this study can explain the capability of the cerebellum to feedforwardly reduce noise effects, such as end-point variability of saccades. This theory assumes that a Golgi-granule cell system can encode the strength of a mossy fiber input as the state of neuronal activity of parallel fibers. By combining these parallel fiber signals with appropriate connection weights to produce a Purkinje cell output, an arbitrary continuous input-output relationship can be obtained. By incorporating such flexible computation and learning ability in a process of saccadic gain adaptation, a new control scheme in which the cerebellar cortex feedforwardly suppresses the end-point variability when it detects a variation in saccadic commands can be devised. Computer simulation confirmed the efficiency of such learning and showed a reduction in the variability of saccadic end points, similar to results obtained from experimental data.

An improved adaptive control for repetitive motion of robots

NASA Technical Reports Server (NTRS)

Pourboghrat, F.

1989-01-01

An adaptive control algorithm is proposed for a class of nonlinear systems, such as robotic manipulators, which is capable of improving its performance in repetitive motions. When the task is repeated, the error between the desired trajectory and that of the system is guaranteed to decrease. The design is based on the combination of a direct adaptive control and a learning process. This method does not require any knowledge of the dynamic parameters of the system.

An intelligent robotic aid system for human services

NASA Technical Reports Server (NTRS)

Kawamura, K.; Bagchi, S.; Iskarous, M.; Pack, R. T.; Saad, A.

1994-01-01

The long term goal of our research at the Intelligent Robotic Laboratory at Vanderbilt University is to develop advanced intelligent robotic aid systems for human services. As a first step toward our goal, the current thrusts of our R&D are centered on the development of an intelligent robotic aid called the ISAC (Intelligent Soft Arm Control). In this paper, we describe the overall system architecture and current activities in intelligent control, adaptive/interactive control and task learning.

Rose garden promises of intelligent tutoring systems: Blossom or thorn

NASA Technical Reports Server (NTRS)

Shute, Valerie J.

1991-01-01

Intelligent tutoring systems (ITS) have been in existence for over a decade. However, few controlled evaluation studies have been conducted comparing the effectiveness of these systems to more traditional instruction methods. Two main promises of ITSs are examined: (1) Engender more effective and efficient learning in relation to traditional formats; and (2) Reduce the range of learning outcome measures where a majority of individuals are elevated to high performance levels. Bloom (1984) has referred to these as the two sigma problem; to achieve two standard deviation improvements with tutoring over traditional instruction methods. Four ITSs are discussed in relation to the two promises. These tutors have undergone systematic, controlled evaluations: (1) The LISP tutor (Anderson Farrell and Sauers, 1984); (2) Smithtown (Shute and Glaser, in press); (3) Sherlock (Lesgold, Lajoie, Bunzo and Eggan, 1990); and (4) The Pascal ITS (Bonar, Cunningham, Beatty and Well, 1988). Results show that these four tutors do accelerate learning with no degradation in final outcome. Suggestions for improvements to the design and evaluation of ITSs are discussed.

Hierarchical HMM based learning of navigation primitives for cooperative robotic endovascular catheterization.

PubMed

Rafii-Tari, Hedyeh; Liu, Jindong; Payne, Christopher J; Bicknell, Colin; Yang, Guang-Zhong

2014-01-01

Despite increased use of remote-controlled steerable catheter navigation systems for endovascular intervention, most current designs are based on master configurations which tend to alter natural operator tool interactions. This introduces problems to both ergonomics and shared human-robot control. This paper proposes a novel cooperative robotic catheterization system based on learning-from-demonstration. By encoding the higher-level structure of a catheterization task as a sequence of primitive motions, we demonstrate how to achieve prospective learning for complex tasks whilst incorporating subject-specific variations. A hierarchical Hidden Markov Model is used to model each movement primitive as well as their sequential relationship. This model is applied to generation of motion sequences, recognition of operator input, and prediction of future movements for the robot. The framework is validated by comparing catheter tip motions against the manual approach, showing significant improvements in the quality of catheterization. The results motivate the design of collaborative robotic systems that are intuitive to use, while reducing the cognitive workload of the operator.

Executive function predicts artificial language learning

PubMed Central

Kapa, Leah L.; Colombo, John

2017-01-01

Previous research suggests executive function (EF) advantages among bilinguals compared to monolingual peers, and these advantages are generally attributed to experience controlling two linguistic systems. However, the possibility that the relationship between bilingualism and EF might be bidirectional has not been widely considered; while experience with two languages might improve EF, better EF skills might also facilitate language learning. In the current studies, we tested whether adults’ and preschool children’s EF abilities predicted success in learning a novel artificial language. After controlling for working memory and English receptive vocabulary, adults’ artificial language performance was predicted by their inhibitory control ability (Study 1) and children’s performance was predicted by their attentional monitoring and shifting ability (Study 2). These findings provide preliminary evidence suggesting that EF processes may be employed during initial stages of language learning, particularly vocabulary acquisition, and support the possibility of a bidirectional relationship between EF and language acquisition. PMID:29129958

Behavioral responses of trained squirrel and rhesus monkeys during oculomotor tasks

PubMed Central

Heiney, Shane A.; Blazquez, Pablo M.

2018-01-01

The oculomotor system is the motor system of choice for many neuroscientists studying motor control and learning because of its simplicity, easy control of inputs (e.g., visual stimulation), and precise control and measurement of motor outputs (eye position). This is especially true in primates, which are easily trained to perform oculomotor tasks. Here we provide the first detailed characterization of the oculomotor performance of trained squirrel monkeys, primates used extensively in oculomotor physiology, during saccade and smooth pursuit tasks, and compare it to that of the rhesus macaque. We found that both primates have similar oculomotor behavior but the rhesus shows a larger oculomotor range, better performance for horizontal saccades above 10 degrees, and better horizontal smooth pursuit gain to target velocities above 15 deg/s. These results are important for interspecies comparisons and necessary when selecting the best stimuli to study motor control and motor learning in the oculomotor systems of these primates. PMID:21656216

Toward a Generative Model of the Teaching-Learning Process.

ERIC Educational Resources Information Center

McMullen, David W.

Until the rise of cognitive psychology, models of the teaching-learning process (TLP) stressed external rather than internal variables. Models remained general descriptions until control theory introduced explicit system analyses. Cybernetic models emphasize feedback and adaptivity but give little attention to creativity. Research on artificial…

Learning-Based Adaptive Optimal Tracking Control of Strict-Feedback Nonlinear Systems.

PubMed

Gao, Weinan; Jiang, Zhong-Ping; Weinan Gao; Zhong-Ping Jiang; Gao, Weinan; Jiang, Zhong-Ping

2018-06-01

This paper proposes a novel data-driven control approach to address the problem of adaptive optimal tracking for a class of nonlinear systems taking the strict-feedback form. Adaptive dynamic programming (ADP) and nonlinear output regulation theories are integrated for the first time to compute an adaptive near-optimal tracker without any a priori knowledge of the system dynamics. Fundamentally different from adaptive optimal stabilization problems, the solution to a Hamilton-Jacobi-Bellman (HJB) equation, not necessarily a positive definite function, cannot be approximated through the existing iterative methods. This paper proposes a novel policy iteration technique for solving positive semidefinite HJB equations with rigorous convergence analysis. A two-phase data-driven learning method is developed and implemented online by ADP. The efficacy of the proposed adaptive optimal tracking control methodology is demonstrated via a Van der Pol oscillator with time-varying exogenous signals.

Distance Learning Methodologies. TRANSCOM Regulating and Command & Control Evacuation System (TRAC2ES).

ERIC Educational Resources Information Center

Bloomquist, Carroll R.

The TRANSCOM (Transportation Command) Regulating Command and Control Evacuation System (TRAC2ES), which applies state-of-the-art technology to manage global medical regulating (matching patients to clinical availability) and medical evacuation processes, will be installed at all Department of Defense medical locations globally. A combination of…

Dealing with Malfunction: Locus of Control in Web-Conferencing

ERIC Educational Resources Information Center

Klebl, Michael

2014-01-01

This paper considers how students deal with malfunctions that occur during the use of web conferencing systems in learning arrangements. In a survey among participants in online courses that make use of a web-conferencing system (N = 129), the relationship between a preference for internal or external locus of control and the perception of…

Deep Learning-Based Data Forgery Detection in Automatic Generation Control

DOE Office of Scientific and Technical Information (OSTI.GOV)

Zhang, Fengli; Li, Qinghua

Automatic Generation Control (AGC) is a key control system in the power grid. It is used to calculate the Area Control Error (ACE) based on frequency and tie-line power flow between balancing areas, and then adjust power generation to maintain the power system frequency in an acceptable range. However, attackers might inject malicious frequency or tie-line power flow measurements to mislead AGC to do false generation correction which will harm the power grid operation. Such attacks are hard to be detected since they do not violate physical power system models. In this work, we propose algorithms based on Neural Networkmore » and Fourier Transform to detect data forgery attacks in AGC. Different from the few previous work that rely on accurate load prediction to detect data forgery, our solution only uses the ACE data already available in existing AGC systems. In particular, our solution learns the normal patterns of ACE time series and detects abnormal patterns caused by artificial attacks. Evaluations on the real ACE dataset show that our methods have high detection accuracy.« less

Controlling a robot with intention derived from motion.

PubMed

Crick, Christopher; Scassellati, Brian

2010-01-01

We present a novel, sophisticated intention-based control system for a mobile robot built from an extremely inexpensive webcam and radio-controlled toy vehicle. The system visually observes humans participating in various playground games and infers their goals and intentions through analyzing their spatiotemporal activity in relation to itself and each other, and then builds a coherent narrative out of the succession of these intentional states. Starting from zero information about the room, the rules of the games, or even which vehicle it controls, it learns rich relationships between players, their goals and intentions, probing uncertain situations with its own behavior. The robot is able to watch people playing various playground games, learn the roles and rules that apply to specific games, and participate in the play. The narratives it constructs capture essential information about the observed social roles and types of activity. After watching play for a short while, the system is able to participate appropriately in the games. We demonstrate how the system acts appropriately in scenarios such as chasing, follow-the-leader, and variants of tag. Copyright © 2009 Cognitive Science Society, Inc.

Automated Subsystem Control for Life Support System (ASCLSS)

NASA Technical Reports Server (NTRS)

Block, Roger F.

1987-01-01

The Automated Subsystem Control for Life Support Systems (ASCLSS) program has successfully developed and demonstrated a generic approach to the automation and control of space station subsystems. The automation system features a hierarchical and distributed real-time control architecture which places maximum controls authority at the lowest or process control level which enhances system autonomy. The ASCLSS demonstration system pioneered many automation and control concepts currently being considered in the space station data management system (DMS). Heavy emphasis is placed on controls hardware and software commonality implemented in accepted standards. The approach demonstrates successfully the application of real-time process and accountability with the subsystem or process developer. The ASCLSS system completely automates a space station subsystem (air revitalization group of the ASCLSS) which moves the crew/operator into a role of supervisory control authority. The ASCLSS program developed over 50 lessons learned which will aide future space station developers in the area of automation and controls..

A composite self tuning strategy for fuzzy control of dynamic systems

NASA Technical Reports Server (NTRS)

Shieh, C.-Y.; Nair, Satish S.

1992-01-01

The feature of self learning makes fuzzy logic controllers attractive in control applications. This paper proposes a strategy to tune the fuzzy logic controller on-line by tuning the data base as well as the rule base. The structure of the controller is outlined and preliminary results are presented using simulation studies.

Interference effects between memory systems in the acquisition of a skill.

PubMed

Gagné, Marie-Hélène; Cohen, Henri

2016-10-01

There is now converging evidence that the declarative memory system (hippocampus dependent) contributes to sequential motor learning in concert with the procedural memory system (striatum dependent). Because of the competition for shared neuronal resources, introducing a declarative memory task can impair learning of a new motor sequence and interference may occur during the procedural consolidation process. Here, we investigated the extent to which interference effects between memory systems are seen at the retrieval phase of skill learning. Healthy participants were assigned to a control (n = 15) or a declarative condition (n = 15) and trained on a sequence of finger movements (FOS task). Both groups showed similar improvement at the end of the practice session on the first day. Twenty-four hours later, controls were tested solely on the FOS task, while subjects in the declarative condition first engaged in a visuospatial task. Additional offline gains in performance were observed only in the control condition. The introduction of a visuospatial memory task just before retrieval of the motor skill was sufficient to eliminate these gains. This suggests that interference between procedural and declarative memory systems may also occur during subsequent motor recall. It is proposed that the interference effects are linked, in part, to the spatial nature of the motor and declarative tasks, which specifically depends upon hippocampal involvement.

Machine Learning: A Crucial Tool for Sensor Design

PubMed Central

Zhao, Weixiang; Bhushan, Abhinav; Santamaria, Anthony D.; Simon, Melinda G.; Davis, Cristina E.

2009-01-01

Sensors have been widely used for disease diagnosis, environmental quality monitoring, food quality control, industrial process analysis and control, and other related fields. As a key tool for sensor data analysis, machine learning is becoming a core part of novel sensor design. Dividing a complete machine learning process into three steps: data pre-treatment, feature extraction and dimension reduction, and system modeling, this paper provides a review of the methods that are widely used for each step. For each method, the principles and the key issues that affect modeling results are discussed. After reviewing the potential problems in machine learning processes, this paper gives a summary of current algorithms in this field and provides some feasible directions for future studies. PMID:20191110

EOL-1, the homolog of the mammalian Dom3Z, regulates olfactory learning in C. elegans.

PubMed

Shen, Yu; Zhang, Jiangwen; Calarco, John A; Zhang, Yun

2014-10-01

Learning is an essential function of the nervous system. However, our understanding of molecular underpinnings of learning remains incomplete. Here, we characterize a conserved protein EOL-1 that regulates olfactory learning in Caenorhabditis elegans. A recessive allele of eol-1 (enhanced olfactory learning) learns better to adjust its olfactory preference for bacteria foods and eol-1 acts in the URX sensory neurons to regulate learning. The mammalian homolog of EOL-1, Dom3Z, which regulates quality control of pre-mRNAs, can substitute the function of EOL-1 in learning regulation, demonstrating functional conservation between these homologs. Mutating the residues of Dom3Z that are critical for its enzymatic activity, and the equivalent residues in EOL-1, abolishes the function of these proteins in learning. Together, our results provide insights into the function of EOL-1/Dom3Z and suggest that its activity in pre-mRNA quality control is involved in neural plasticity. Copyright © 2014 the authors 0270-6474/14/3413364-07$15.00/0.

Building environment analysis based on temperature and humidity for smart energy systems.

PubMed

Yun, Jaeseok; Won, Kwang-Ho

2012-10-01

In this paper, we propose a new HVAC (heating, ventilation, and air conditioning) control strategy as part of the smart energy system that can balance occupant comfort against building energy consumption using ubiquitous sensing and machine learning technology. We have developed ZigBee-based wireless sensor nodes and collected realistic temperature and humidity data during one month from a laboratory environment. With the collected data, we have established a building environment model using machine learning algorithms, which can be used to assess occupant comfort level. We expect the proposed HVAC control strategy will be able to provide occupants with a consistently comfortable working or home environment.

Self-Learning Power Control in Wireless Sensor Networks.

PubMed

Chincoli, Michele; Liotta, Antonio

2018-01-27

Current trends in interconnecting myriad smart objects to monetize on Internet of Things applications have led to high-density communications in wireless sensor networks. This aggravates the already over-congested unlicensed radio bands, calling for new mechanisms to improve spectrum management and energy efficiency, such as transmission power control. Existing protocols are based on simplistic heuristics that often approach interference problems (i.e., packet loss, delay and energy waste) by increasing power, leading to detrimental results. The scope of this work is to investigate how machine learning may be used to bring wireless nodes to the lowest possible transmission power level and, in turn, to respect the quality requirements of the overall network. Lowering transmission power has benefits in terms of both energy consumption and interference. We propose a protocol of transmission power control through a reinforcement learning process that we have set in a multi-agent system. The agents are independent learners using the same exploration strategy and reward structure, leading to an overall cooperative network. The simulation results show that the system converges to an equilibrium where each node transmits at the minimum power while respecting high packet reception ratio constraints. Consequently, the system benefits from low energy consumption and packet delay.

Self-Learning Power Control in Wireless Sensor Networks

PubMed Central

Liotta, Antonio

2018-01-01

Current trends in interconnecting myriad smart objects to monetize on Internet of Things applications have led to high-density communications in wireless sensor networks. This aggravates the already over-congested unlicensed radio bands, calling for new mechanisms to improve spectrum management and energy efficiency, such as transmission power control. Existing protocols are based on simplistic heuristics that often approach interference problems (i.e., packet loss, delay and energy waste) by increasing power, leading to detrimental results. The scope of this work is to investigate how machine learning may be used to bring wireless nodes to the lowest possible transmission power level and, in turn, to respect the quality requirements of the overall network. Lowering transmission power has benefits in terms of both energy consumption and interference. We propose a protocol of transmission power control through a reinforcement learning process that we have set in a multi-agent system. The agents are independent learners using the same exploration strategy and reward structure, leading to an overall cooperative network. The simulation results show that the system converges to an equilibrium where each node transmits at the minimum power while respecting high packet reception ratio constraints. Consequently, the system benefits from low energy consumption and packet delay. PMID:29382072

Design and evaluation of a sensor fail-operational control system for a digitally controlled turbofan engine

NASA Technical Reports Server (NTRS)

Hrach, F. J.; Arpasi, D. J.; Bruton, W. M.

1975-01-01

A self-learning, sensor fail-operational, control system for the TF30-P-3 afterburning turbofan engine was designed and evaluated. The sensor fail-operational control system includes a digital computer program designed to operate in conjunction with the standard TF30-P-3 bill-of-materials control. Four engine measurements and two compressor face measurements are tested. If any engine measurements are found to have failed, they are replaced by values synthesized from computer-stored information. The control system was evaluated by using a realtime, nonlinear, hybrid computer engine simulation at sea level static condition, at a typical cruise condition, and at several extreme flight conditions. Results indicate that the addition of such a system can improve the reliability of an engine digital control system.

Functional Based Adaptive and Fuzzy Sliding Controller for Non-Autonomous Active Suspension System

NASA Astrophysics Data System (ADS)

Huang, Shiuh-Jer; Chen, Hung-Yi

In this paper, an adaptive sliding controller is developed for controlling a vehicle active suspension system. The functional approximation technique is employed to substitute the unknown non-autonomous functions of the suspension system and release the model-based requirement of sliding mode control algorithm. In order to improve the control performance and reduce the implementation problem, a fuzzy strategy with online learning ability is added to compensate the functional approximation error. The update laws of the functional approximation coefficients and the fuzzy tuning parameters are derived from the Lyapunov theorem to guarantee the system stability. The proposed controller is implemented on a quarter-car hydraulic actuating active suspension system test-rig. The experimental results show that the proposed controller suppresses the oscillation amplitude of the suspension system effectively.

Observer-based distributed adaptive iterative learning control for linear multi-agent systems

NASA Astrophysics Data System (ADS)

Li, Jinsha; Liu, Sanyang; Li, Junmin

2017-10-01

This paper investigates the consensus problem for linear multi-agent systems from the viewpoint of two-dimensional systems when the state information of each agent is not available. Observer-based fully distributed adaptive iterative learning protocol is designed in this paper. A local observer is designed for each agent and it is shown that without using any global information about the communication graph, all agents achieve consensus perfectly for all undirected connected communication graph when the number of iterations tends to infinity. The Lyapunov-like energy function is employed to facilitate the learning protocol design and property analysis. Finally, simulation example is given to illustrate the theoretical analysis.

Fuzzy Counter Propagation Neural Network Control for a Class of Nonlinear Dynamical Systems

PubMed Central

Sakhre, Vandana; Jain, Sanjeev; Sapkal, Vilas S.; Agarwal, Dev P.

2015-01-01

Fuzzy Counter Propagation Neural Network (FCPN) controller design is developed, for a class of nonlinear dynamical systems. In this process, the weight connecting between the instar and outstar, that is, input-hidden and hidden-output layer, respectively, is adjusted by using Fuzzy Competitive Learning (FCL). FCL paradigm adopts the principle of learning, which is used to calculate Best Matched Node (BMN) which is proposed. This strategy offers a robust control of nonlinear dynamical systems. FCPN is compared with the existing network like Dynamic Network (DN) and Back Propagation Network (BPN) on the basis of Mean Absolute Error (MAE), Mean Square Error (MSE), Best Fit Rate (BFR), and so forth. It envisages that the proposed FCPN gives better results than DN and BPN. The effectiveness of the proposed FCPN algorithms is demonstrated through simulations of four nonlinear dynamical systems and multiple input and single output (MISO) and a single input and single output (SISO) gas furnace Box-Jenkins time series data. PMID:26366169

Fuzzy Counter Propagation Neural Network Control for a Class of Nonlinear Dynamical Systems.

PubMed

Sakhre, Vandana; Jain, Sanjeev; Sapkal, Vilas S; Agarwal, Dev P

2015-01-01

Fuzzy Counter Propagation Neural Network (FCPN) controller design is developed, for a class of nonlinear dynamical systems. In this process, the weight connecting between the instar and outstar, that is, input-hidden and hidden-output layer, respectively, is adjusted by using Fuzzy Competitive Learning (FCL). FCL paradigm adopts the principle of learning, which is used to calculate Best Matched Node (BMN) which is proposed. This strategy offers a robust control of nonlinear dynamical systems. FCPN is compared with the existing network like Dynamic Network (DN) and Back Propagation Network (BPN) on the basis of Mean Absolute Error (MAE), Mean Square Error (MSE), Best Fit Rate (BFR), and so forth. It envisages that the proposed FCPN gives better results than DN and BPN. The effectiveness of the proposed FCPN algorithms is demonstrated through simulations of four nonlinear dynamical systems and multiple input and single output (MISO) and a single input and single output (SISO) gas furnace Box-Jenkins time series data.

Actor-critic-based optimal tracking for partially unknown nonlinear discrete-time systems.

PubMed

Kiumarsi, Bahare; Lewis, Frank L

2015-01-01

This paper presents a partially model-free adaptive optimal control solution to the deterministic nonlinear discrete-time (DT) tracking control problem in the presence of input constraints. The tracking error dynamics and reference trajectory dynamics are first combined to form an augmented system. Then, a new discounted performance function based on the augmented system is presented for the optimal nonlinear tracking problem. In contrast to the standard solution, which finds the feedforward and feedback terms of the control input separately, the minimization of the proposed discounted performance function gives both feedback and feedforward parts of the control input simultaneously. This enables us to encode the input constraints into the optimization problem using a nonquadratic performance function. The DT tracking Bellman equation and tracking Hamilton-Jacobi-Bellman (HJB) are derived. An actor-critic-based reinforcement learning algorithm is used to learn the solution to the tracking HJB equation online without requiring knowledge of the system drift dynamics. That is, two neural networks (NNs), namely, actor NN and critic NN, are tuned online and simultaneously to generate the optimal bounded control policy. A simulation example is given to show the effectiveness of the proposed method.

Near Optimal Event-Triggered Control of Nonlinear Discrete-Time Systems Using Neurodynamic Programming.

PubMed

Sahoo, Avimanyu; Xu, Hao; Jagannathan, Sarangapani

2016-09-01

This paper presents an event-triggered near optimal control of uncertain nonlinear discrete-time systems. Event-driven neurodynamic programming (NDP) is utilized to design the control policy. A neural network (NN)-based identifier, with event-based state and input vectors, is utilized to learn the system dynamics. An actor-critic framework is used to learn the cost function and the optimal control input. The NN weights of the identifier, the critic, and the actor NNs are tuned aperiodically once every triggered instant. An adaptive event-trigger condition to decide the trigger instants is derived. Thus, a suitable number of events are generated to ensure a desired accuracy of approximation. A near optimal performance is achieved without using value and/or policy iterations. A detailed analysis of nontrivial inter-event times with an explicit formula to show the reduction in computation is also derived. The Lyapunov technique is used in conjunction with the event-trigger condition to guarantee the ultimate boundedness of the closed-loop system. The simulation results are included to verify the performance of the controller. The net result is the development of event-driven NDP.

Practice and effectiveness of web-based problem-based learning approach in a large class-size system: A comparative study.

PubMed

Ding, Yongxia; Zhang, Peili

2018-06-12

Problem-based learning (PBL) is an effective and highly efficient teaching approach that is extensively applied in education systems across a variety of countries. This study aimed to investigate the effectiveness of web-based PBL teaching pedagogies in large classes. The cluster sampling method was used to separate two college-level nursing student classes (graduating class of 2013) into two groups. The experimental group (n = 162) was taught using a web-based PBL teaching approach, while the control group (n = 166) was taught using conventional teaching methods. We subsequently assessed the satisfaction of the experimental group in relation to the web-based PBL teaching mode. This assessment was performed following comparison of teaching activity outcomes pertaining to exams and self-learning capacity between the two groups. When compared with the control group, the examination scores and self-learning capabilities were significantly higher in the experimental group (P < 0.01) compared with the control group. In addition, 92.6% of students in the experimental group expressed satisfaction with the new web-based PBL teaching approach. In a large class-size teaching environment, the web-based PBL teaching approach appears to be more optimal than traditional teaching methods. These results demonstrate the effectiveness of web-based teaching technologies in problem-based learning. Copyright © 2018. Published by Elsevier Ltd.