Abstract: A control apparatus and methods using context-dependent difference learning for controlling e.g., a plant. In one embodiment, the apparatus includes an actor module and a critic module. The actor module provides a control signal for the plant. The actor module is subject to adaptation, which is performed to optimize control strategy of the actor. The adaptation is based upon the reinforcement signal provided by the critic module. The reinforcement signal is calculated based on the comparison of a present control performance signal observed for a certain context signal, with a control performance signal observed for the same context in the past.

Abstract: A robotic device may comprise an adaptive controller configured to learn to predict consequences of robotic device's actions. During training, the controller may receive a copy of the planned and/or executed motor command and sensory information obtained based on the robot's response to the command. The controller may predict sensory outcome based on the command and one or more prior sensory inputs. The predicted sensory outcome may be compared to the actual outcome. Based on a determination that the prediction matches the actual outcome, the training may stop. Upon detecting a discrepancy between the prediction and the actual outcome, the controller may provide a continuation signal configured to indicate that additional training may be utilized. In some classification implementations, the discrepancy signal may be used to indicate occurrence of novel (not yet learned) objects in the sensory input and/or indicate continuation of training to recognize said objects.

Abstract: Robotic devices may be trained by a trainer guiding the robot along a target trajectory using physical contact with the robot. The robot may comprise an adaptive controller configured to generate control commands based on one or more of the trainer input, sensory input, and/or performance measure. The trainer may observe task execution by the robot. Responsive to observing a discrepancy between the target behavior and the actual behavior, the trainer may provide a teaching input via a haptic action. The robot may execute the action based on a combination of the internal control signal produced by a learning process of the robot and the training input. The robot may infer the teaching input based on a comparison of a predicted state and actual state of the robot. The robot's learning process may be adjusted in accordance with the teaching input so as to reduce the discrepancy during a subsequent trial.

Abstract: Framework may be implemented for transferring knowledge from an external agent to a robotic controller. In an obstacle avoidance/target approach application, the controller may be configured to determine a teaching signal based on a sensory input, the teaching signal conveying information associated with target action consistent with the sensory input, the sensory input being indicative of the target/obstacle. The controller may be configured to determine a control signal based on the sensory input, the control signal conveying information associated with target approach/avoidance action. The controller may determine a predicted control signal based on the sensory input and the teaching signal, the predicted control conveying information associated with the target action. The control signal may be combined with the predicted control in order to cause the robotic apparatus to execute the target action.

Abstract: State-dependent supervised learning framework in artificial neuron networks may be implemented. A framework may be used to describe plasticity updates of neuron connections based on a connection state term and a neuron state term. Connection states may be updated based on inputs and outputs to and/or from neurons. The input connections of a neuron may be updated using input traces comprising a time-history of inputs provided via the connection. Weight of the connection may be updated and connection state may be time varying. The updated weights may be determined using a rate of change of the input trace and a term comprising a product of a per-neuron contribution and a per-connection contribution configured to account for the state time-dependency. Using event-dependent connection change components, connection updates may be executed on a per neuron basis, as opposed to a per-connection basis.

Abstract: Neural network apparatus and methods for implementing reinforcement learning. In one implementation, the neural network is a spiking neural network, and the apparatus and methods may be used for example to enable an adaptive signal processing system to effect focused exploration by associative adaptation, including providing a negative reward signal to the network, which may increase excitability of the neurons in combination with decrease in excitability of active neurons. In certain implementations, the increase is gradual and of smaller magnitude, compared to the excitability decrease. In some implementations, the increase/decrease of the neuron excitability is effectuated by increasing/decreasing an efficacy of the respective synaptic connections delivering presynaptic inputs into the neuron. The focused exploration may be achieved for instance by non-associative potentiation configured based at least on the input spike rate.

Abstract: An action for execution by a robotic device may be selected. A robotic controller may determine that two or more actions are to be executed based on analysis of sensory and/or training input. The actions may comprise target approach and/or obstacle avoidance. Execution of individual actions may be based on a control signal and a separate activation signal being generated by the controller. Control signal execution may be inhibited by the controller relay block. Multiple activation signals may compete with one another in winner-take-all action selection network to produce selection signal. The selection signal may temporarily pause inhibition of a respective portion of the relay block that is associated with the winning activation signal channel. A disinhibited portion of the relay block may provide the respective control signal for execution by a controllable element. Arbitration between individual actions may be performed based on evaluation of activation signals.

Abstract: Apparatus and methods for training of robotic devices. Robotic devices may be trained by a user guiding the robot along target trajectory using an input signal. A robotic device may comprise an adaptive controller configured to generate control commands based on one or more of the user guidance, sensory input, and/or performance measure. Training may comprise a plurality of trials. During first trial, the user input may be sufficient to cause the robot to complete the trajectory. During subsequent trials, the user and the robot's controller may collaborate so that user input may be reduced while the robot control may be increased. Individual contributions from the user and the robot controller during training may be may be inadequate (when used exclusively) to complete the task.

Abstract: A robot may be trained by a user guiding the robot along target trajectory using a control signal. A robot may comprise an adaptive controller. The controller may be configured to generate control commands based on the user guidance, sensory input and a performance measure. A user may interface to the robot via an adaptively configured remote controller. The remote controller may comprise a mobile device, configured by the user in accordance with phenotype and/or operational configuration of the robot. The remote controller may detect changes in the robot phenotype and/or operational configuration. The remote controller may comprise multiple control elements configured to activate respective portions of the robot platform. Based on training, the remote controller may configure composite controls configured based two or more of control elements. Activation of a composite control may enable the robot to perform a task.

Abstract: Apparatus and methods for universal node design implementing a universal learning rule in a mixed signal spiking neural network. In one implementation, at one instance, the node apparatus, operable according to the parameterized universal learning model, receives a mixture of analog and spiking inputs, and generates a spiking output based on the model parameter for that node that is selected by the parameterized model for that specific mix of inputs. At another instance, the same node receives a different mix of inputs, that also may comprise only analog or only spiking inputs and generates an analog output based on a different value of the node parameter that is selected by the model for the second mix of inputs. In another implementation, the node apparatus may change its output from analog to spiking responsive to a training input for the same inputs.

Abstract: Generalized state-dependent learning framework in artificial neuron networks may be implemented. A framework may be used to describe plasticity updates of neuron connections based on connection state term and neuron state term. The state connections within the network may be updated based on inputs and outputs to/from neurons. The input connections of a neuron may be updated using connection traces comprising a time-history of inputs provided via the connections. Weights of the connections may be updated and connection state may be time varying. The updated weights may be determined using a rate of change of the trace and a term comprising a product of a per-neuron contribution and a per-connection contribution configured to account for the state time-dependency. Using event-dependent connection change components, connection updates may be executed on per neuron basis, as opposed to per-connection basis.

Abstract: Neural network apparatus and methods for implementing reinforcement learning. In one implementation, the neural network is a spiking neural network, and the apparatus and methods may be used for example to enable an adaptive signal processing system to effect network adaptation by optimized credit assignment. In certain implementations, the credit assignment may be based on a comparison between network output and individual unit contribution. The unit contribution may be determined for example using eligibility traces that may comprise pre-synaptic and/or post-synaptic activity. In certain implementations, the unit credit may be determined using correlation between rate of change of network output and eligibility trace of the unit.

Abstract: Neural network apparatus and methods for implementing reinforcement learning. In one implementation, the neural network is a spiking neural network, and the apparatus and methods may be used for example to enable an adaptive signal processing system to effect focused exploration by associative adaptation, including providing a negative reward signal to the network, which may increase excitability of the neurons in combination with decrease in excitability of active neurons. In certain implementations, the increase is gradual and of smaller magnitude, compared to the excitability decrease. In some implementations, the increase/decrease of the neuron excitability is effectuated by increasing/decreasing an efficacy of the respective synaptic connections delivering presynaptic inputs into the neuron. The focused exploration may be achieved for instance by non-associative potentiation configured based at least on the input spike rate.

Abstract: Apparatus and methods for universal node design implementing a universal learning rule in a mixed signal spiking neural network. In one implementation, at one instance, the node apparatus, operable according to the parameterized universal learning model, receives a mixture of analog and spiking inputs, and generates a spiking output based on the model parameter for that node that is selected by the parameterized model for that specific mix of inputs. At another instance, the same node receives a different mix of inputs, that also may comprise only analog or only spiking inputs and generates an analog output based on a different value of the node parameter that is selected by the model for the second mix of inputs. In another implementation, the node apparatus may change its output from analog to spiking responsive to a training input for the same inputs.

Abstract: Apparatus and methods for universal node design implementing a universal learning rule in a mixed signal spiking neural network. In one implementation, at one instance, the node apparatus, operable according to the parameterized universal learning model, receives a mixture of analog and spiking inputs, and generates a spiking output based on the model parameter for that node that is selected by the parameterized model for that specific mix of inputs. At another instance, the same node receives a different mix of inputs, that also may comprise only analog or only spiking inputs and generates an analog output based on a different value of the node parameter that is selected by the model for the second mix of inputs. In another implementation, the node apparatus may change its output from analog to spiking responsive to a training input for the same inputs.

Abstract: Apparatus and methods for universal node design implementing a universal learning rule in a mixed signal spiking neural network. In one implementation, at one instance, the node apparatus, operable according to the parameterized universal learning model, receives a mixture of analog and spiking inputs, and generates a spiking output based on the model parameter for that node that is selected by the parameterized model for that specific mix of inputs. At another instance, the same node receives a different mix of inputs, that also may comprise only analog or only spiking inputs and generates an analog output based on a different value of the node parameter that is selected by the model for the second mix of inputs. In another implementation, the node apparatus may change its output from analog to spiking responsive to a training input for the same inputs.

Abstract: A control apparatus and methods using context-dependent difference learning for controlling e.g., a plant. In one embodiment, the apparatus includes an actor module and a critic module. The actor module provides a control signal for the plant. The actor module is subject to adaptation, which is performed to optimize control strategy of the actor. The adaptation is based upon the reinforcement signal provided by the critic module. The reinforcement signal is calculated based on the comparison of a present control performance signal observed for a certain context signal, with a control performance signal observed for the same context in the past.