Abstract: A method for localizing and estimating a pose of a known object in a field of view of a vision system is described, and includes developing a processor-based model of the known object, capturing a bitmap image file including an image of the field of view including the known object, extracting features from the bitmap image file, matching the extracted features with features associated with the model of the known object, localizing an object in the bitmap image file based upon the extracted features, clustering the extracted features of the localized object, merging the clustered extracted features, detecting the known object in the field of view based upon a comparison of the merged clustered extracted features and the processor-based model of the known object, and estimating a pose of the detected known object in the field of view based upon the detecting of the known object.

Abstract: A method for localizing and estimating a pose of a known object in a field of view of a vision system is described, and includes developing a processor-based model of the known object, capturing a bitmap image file including an image of the field of view including the known object, extracting features from the bitmap image file, matching the extracted features with features associated with the model of the known object, localizing an object in the bitmap image file based upon the extracted features, clustering the extracted features of the localized object, merging the clustered extracted features, detecting the known object in the field of view based upon a comparison of the merged clustered extracted features and the processor-based model of the known object, and estimating a pose of the detected known object in the field of view based upon the detecting of the known object.

Abstract: A robotic system includes a robot, sensors which measure status information including a position and orientation of the robot and an object within the workspace, and a controller. The controller, which visually debugs an operation of the robot, includes a simulator module, action planning module, and graphical user interface (GUI). The simulator module receives the status information and generates visual markers, in response to marker commands, as graphical depictions of the object and robot. An action planning module selects a next action of the robot. The marker generator module generates and outputs the marker commands to the simulator module in response to the selected next action. The GUI receives and displays the visual markers, selected future action, and input commands. Via the action planning module, the position and/or orientation of the visual markers are modified in real time to change the operation of the robot.

Abstract: A method of training a robotic device to perform predetermined movements utilizing wearable sensors worn by a user. A position and movement of the user is sensed in response to signals from the wearable sensors. The wearable sensors include at least one six-degree of freedom accelerometer. Gestures are recognized by a gesture recognizer device in response to the signals. A position, orientation, and velocity of the user are estimated by a position and orientation processing unit. The sensed gesture inputs and the estimated position, orientation, and velocity inputs of the wearable sensors are received within a service requester device. The gestures, orientations, positions, and velocities are converted into predefined actions. Controls signals are output from the service requester device to a robot controller for executing the predefined actions.

Abstract: A robotic system for performing an autonomous task includes a humanoid robot having a plurality of compliant robotic joints, actuators, and other integrated system devices that are controllable in response to control data from various control points, and having sensors for measuring feedback data at the control points. The system includes a multi-level distributed control framework (DCF) for controlling the integrated system components over multiple high-speed communication networks. The DCF has a plurality of first controllers each embedded in a respective one of the integrated system components, e.g., the robotic joints, a second controller coordinating the components via the first controllers, and a third controller for transmitting a signal commanding performance of the autonomous task to the second controller. The DCF virtually centralizes all of the control data and the feedback data in a single location to facilitate control of the robot across the multiple communication networks.

Abstract: A robotic system includes a robot, sensors which measure status information including a position and orientation of the robot and an object within the workspace, and a controller. The controller, which visually debugs an operation of the robot, includes a simulator module, action planning module, and graphical user interface (GUI). The simulator module receives the status information and generates visual markers, in response to marker commands, as graphical depictions of the object and robot. An action planning module selects a next action of the robot. The marker generator module generates and outputs the marker commands to the simulator module in response to the selected next action. The GUI receives and displays the visual markers, selected future action, and input commands. Via the action planning module, the position and/or orientation of the visual markers are modified in real time to change the operation of the robot.

Abstract: A method of training a robotic device to perform predetermined movements utilizing wearable sensors worn by a user. A position and movement of the user is sensed in response to signals from the wearable sensors. The wearable sensors include at least one six-degree of freedom accelerometer. Gestures are recognized by a gesture recognizer device in response to the signals. A position, orientation, and velocity of the user are estimated by a position and orientation processing unit. The sensed gesture inputs and the estimated position, orientation, and velocity inputs of the wearable sensors are received within a service requester device. The gestures, orientations, positions, and velocities are converted into predefined actions. Controls signals are output from the service requester device to a robot controller for executing the predefined actions.

Abstract: A method for training a robot to execute a robotic task in a work environment includes moving the robot across its configuration space through multiple states of the task and recording motor schema describing a sequence of behavior of the robot. Sensory data describing performance and state values of the robot is recorded while moving the robot. The method includes detecting perceptual features of objects located in the environment, assigning virtual deictic markers to the detected perceptual features, and using the assigned markers and the recorded motor schema to subsequently control the robot in an automated execution of another robotic task. Markers may be combined to produce a generalized marker. A system includes the robot, a sensor array for detecting the performance and state values, a perceptual sensor for imaging objects in the environment, and an electronic control unit that executes the present method.

Abstract: Methods and apparatus for procedural memory learning to control a robot by demonstrating a task action to the robot and having the robot learn the action according to a similarity matrix of correlated values, attributes, and parameters obtained from the robot as the robot performs the demonstrated action. Learning is done by an artificial neural network associated with the robot controller, so that the robot learns to perform the task associated with the similarity matrix. Extended similarity matrices can contain integrated and differentiated values of variables. Procedural memory learning reduces overhead in instructing robots to perform tasks. Continued learning improves performance and provides automatic compensation for changes in robot condition and environmental factors.

Abstract: A method and system for characterizing, detecting, and predicting or forecasting multiple target events from a past history of these events includes compressing temporal data streams into self-organizing map (SOM) clusters, and determining trajectories of the temporal streams via the clusters to predict the multiple target events. The system includes an evolutionary multi-objective optimization (EMO) module for processing the temporal data streams, which are obtained from a plurality of heterogeneous domains; a SOM module for characterizing the temporal data streams into self-organizing map clusters; and a target event prediction (TEP) module for generating prediction models of the map clusters. The SOM module employs a vector quantization method that places a set of vectors on a low-dimensional grid in an ordered fashion. The prediction models each include trajectories of the temporal data streams, and the system predicts the multiple target events using the trajectories.

Abstract: A method for training a robot to execute a robotic task in a work environment includes moving the robot across its configuration space through multiple states of the task and recording motor schema describing a sequence of behavior of the robot. Sensory data describing performance and state values of the robot is recorded while moving the robot. The method includes detecting perceptual features of objects located in the environment, assigning virtual deictic markers to the detected perceptual features, and using the assigned markers and the recorded motor schema to subsequently control the robot in an automated execution of another robotic task. Markers may be combined to produce a generalized marker. A system includes the robot, a sensor array for detecting the performance and state values, a perceptual sensor for imaging objects in the environment, and an electronic control unit that executes the present method.

Abstract: Methods and apparatus for procedural memory learning to control a robot by demonstrating a task action to the robot and having the robot learn the action according to a similarity matrix of correlated values, attributes, and parameters obtained from the robot as the robot performs the demonstrated action. Learning is done by an artificial neural network associated with the robot controller, so that the robot learns to perform the task associated with the similarity matrix. Extended similarity matrices can contain integrated and differentiated values of variables. Procedural memory learning reduces overhead in instructing robots to perform tasks. Continued learning improves performance and provides automatic compensation for changes in robot condition and environmental factors.

Abstract: A system associated with handling an object with a gripper includes a sensor that is configured to measure spatially distributed data that represents the position of the object that is handled by the gripper. The system further includes a computing unit that is configured to determine the behavior of the object.

Abstract: A robotic system includes a humanoid robot with multiple compliant joints, each moveable using one or more of the actuators, and having sensors for measuring control and feedback data. A distributed controller controls the joints and other integrated system components over multiple high-speed communication networks. Diagnostic, prognostic, and health management (DPHM) modules are embedded within the robot at the various control levels. Each DPHM module measures, controls, and records DPHM data for the respective control level/connected device in a location that is accessible over the networks or via an external device. A method of controlling the robot includes embedding a plurality of the DPHM modules within multiple control levels of the distributed controller, using the DPHM modules to measure DPHM data within each of the control levels, and recording the DPHM data in a location that is accessible over at least one of the high-speed communication networks.

Abstract: A method is provided for determining when to provide a refueling notification to a driver of a vehicle. A refueling behavior is determined for refueling the vehicle. The refueling behavior is associated at least in part to an amount of fuel customarily remaining in the vehicle when the vehicle is customarily refueled. A remaining amount of fuel in the vehicle and a fuel economy of the vehicle are determined. A distance the vehicle will travel to a next driving destination is estimated. An amount of fuel that will be used to travel to the next driving destination is estimated based on the estimated distance the vehicle will travel to the next driving destination and the fuel economy. A determination is made whether the amount of fuel that will be remaining in the vehicle after the vehicle travels to the next driving destination is less than the amount of a fuel customarily remaining in the vehicle when the vehicle is refueled.

Abstract: An integrated real and virtual manufacturing automation system that employs a programmable logic controller that controls part flow between a real machine in the real world part of the system and a virtual machine in the virtual world part of the system using virtually coupled sensors and actuators. A real world sensor senses the position of the real world machine and a real world actuator actuates the real world machine. Likewise, a virtual world sensor senses the position of the virtual world machine and a virtual world actuator actuates the virtual world machine. An interface device transfers signals between the virtual world part of the system and the real world part of the system, and an input/output device processes signals sent to the programmable logic controller and signals sent from the programmable logic controller.

Abstract: A system associated with handling an object with a gripper includes a sensor that is configured to measure spatially distributed data that represents the position of the object that is handled by the gripper. The system further includes a computing unit that is configured to determine the behavior of the object.

Abstract: A method and system for characterizing, detecting, and predicting or forecasting multiple target events from a past history of these events includes compressing temporal data streams into self-organizing map (SOM) clusters, and determining trajectories of the temporal streams via the clusters to predict the multiple target events. The system includes an evolutionary multi-objective optimization (EMO) module for processing the temporal data streams, which are obtained from a plurality of heterogeneous domains; a SOM module for characterizing the temporal data streams into self-organizing map clusters; and a target event prediction (TEP) module for generating prediction models of the map clusters. The SOM module employs a vector quantization method that places a set of vectors on a low-dimensional grid in an ordered fashion. The prediction models each include trajectories of the temporal data streams, and the system predicts the multiple target events using the trajectories.

Abstract: A robotic system for performing an autonomous task includes a humanoid robot having a plurality of compliant robotic joints, actuators, and other integrated system devices that are controllable in response to control data from various control points, and having sensors for measuring feedback data at the control points. The system includes a multi-level distributed control framework (DCF) for controlling the integrated system components over multiple high-speed communication networks. The DCF has a plurality of first controllers each embedded in a respective one of the integrated system components, e.g., the robotic joints, a second controller coordinating the components via the first controllers, and a third controller for transmitting a signal commanding performance of the autonomous task to the second controller. The DCF virtually centralizes all of the control data and the feedback data in a single location to facilitate control of the robot across the multiple communication networks.

Abstract: A robotic system includes a humanoid robot with multiple compliant joints, each moveable using one or more of the actuators, and having sensors for measuring control and feedback data. A distributed controller controls the joints and other integrated system components over multiple high-speed communication networks. Diagnostic, prognostic, and health management (DPHM) modules are embedded within the robot at the various control levels. Each DPHM module measures, controls, and records DPHM data for the respective control level/connected device in a location that is accessible over the networks or via an external device. A method of controlling the robot includes embedding a plurality of the DPHM modules within multiple control levels of the distributed controller, using the DPHM modules to measure DPHM data within each of the control levels, and recording the DPHM data in a location that is accessible over at least one of the high-speed communication networks.