Abstract: Objects having a flat surface such as a table are detected by processing a depth image and a color image. A mask indicating an area likely to include an object having the flat surface is generated by processing a depth image including the depth information. A color image corresponding to the depth image is then cropped using the mask to detect a portion of the color image that likely include the object having the flat surface. Geometric features of the cropped color image such as lines are then detected to determine the location and orientation of the object having the flat surface. A subset of the detected geometric features is selected as outlines of the flat surface.

Abstract: A system or method for generating gestures in a robot during generation of a speech output by the robot by analyzing a speech text and selecting appropriate gestures from a plurality of candidate gestures. The speech text is analyzed and tagged with information relevant to generating of the gestures. Based on the speech text, the tagged information and other relevant information, a gesture identifier is selected. A gesture template corresponding to the gesture identifier is retrieved and then processed by adding relevant parameter to generate a gesture descriptor representing a gesture to be taken by the robot. A gesture motion is planned based on the gesture descriptor and analysis of timing associated with the speech. Actuator signals for controlling the actuators such as arms and hands are generated based on the planned gesture motion.

Abstract: Objects placed on a flat surface are identified and localized by using a single view image. The single view image in the perspective projection is transformed to a normalized image in a pseudo plan to view to enhance detection of the bottom or top shapes of the objects. One or more geometric features are detected from the normalized image by processing the normalized image. The detected geometric features are analyzed to determine the identity and the location the objects on the flat surface.

Abstract: Multi-modal planning method and system that search a path for the most constrained mode first, and then expands the searches for path in a less constrained mode. By searching the path for the most constrained mode first, less resource are wasted on searching for paths that does not result in a feasible path in the most constrained mode. Multi-modal planning is performed by precomputing feasibility and utility of transition configurations of two adjacent modes. The feasibility is used to exclude non-feasible transition configurations in the most constrained mode from being sampled. The utility is used to bias sampling of the transition configuration so that transition configurations with higher utility are sampled with higher probability. Paths of configurations with higher utility and efficiency are obtained by biasing the sampling of the transition configurations.

Abstract: Systems and methods are presented that enable a higher-level software application to control a robot's motion through a generic motion interface. In one embodiment, a system includes a controller, an interface, and a set of robot driver modules. The interface receives a command from the controller and translates the command into another command to send to the driver modules. The interface includes a client, a server, and a network. The server includes two interfaces: a client interface to communicate with the client and a driver interface to communicate with the driver modules. The server also includes two buffers: a command queue and a reply queue. The command queue stores commands received from the controller (via the client). The reply queue stores replies received from the driver modules.

Abstract: An efficient technique to build subject-specific skeleton models from external measurements is provided. A generic human skeleton model is manipulated and deformed using marker positional data from human motion-captured sequences. The joint and bone geometry parameters are optimized to construct an estimate of the specific skeleton model of the human subject that originally performed the motion sequence. The scope of the fitting procedure can be adjusted to coincide with a series of smaller optimizations over local bone regions or expanded to a single global optimization over all skeleton geometry and joint parameters simultaneously.

Abstract: A robot using less storage and computational resources to embody panoramic attention. The robot includes a panoramic attention module with multiple levels that are hierarchically structured to process different levels of information. The top-level of the panoramic attention module receives information about entities detected from the environment of the robot and maps the entities to a panoramic map maintained by the robot. By mapping and storing high-level entity information instead of low-level sensory information in the panoramic map, the amount of storage and computation resources for panoramic attention can be reduced significantly. Further, the mapping and storing of high-level entity information in the panoramic map also facilitates consistent and logical processing of different conceptual levels of information.

Abstract: Systems and methods are presented that enable robot commands to be determined based on the pose of an object. A method is described for determining the orientation and pose of an object using indistinguishable points. The resolution of the pose detection is based on the robot'command vocabulary. A system is described for controlling a robot by pose detection using unlabelled points.

Abstract: Objects having a flat surface such as a table are detected by processing a depth image and a color image. A mask indicating an area likely to include an object having the flat surface is generated by processing a depth image including the depth information. A color image corresponding to the depth image is then cropped using the mask to detect a portion of the color image that likely include the object having the flat surface. Geometric features of the cropped color image such as lines are then detected to determine the location and orientation of the object having the flat surface. A subset of the detected geometric features is selected as outlines of the flat surface.

Abstract: A system and method for animating a character with activation-driven muscle deformation. External loads can be estimated through an iterative joint torque estimation process, and the external loads reflected in a physical model. Kinematic motion and the physical model reflecting external loads can be used to estimate joint torques. Muscle activations can be determined from the joint torques, and a character can be animated with muscle deformation responsive to the muscle activations.

Abstract: A publish-subscribe architecture based blackboard system for selecting and exchanging selected information among a plurality of processing modules using filters for implementing conditions described in a procedural language to reduce the amount of information transmitted between the processing modules. More than one filter may be dynamically installed in a message hub to select and collect the published information to be sent to a certain subscribing module. By using the procedural language to describe the filters, the message hub can more intelligently select the information to be sent to the subscribing module. This reduces the amount of information transmitted via communication channels. Further, the subscribing module may be relieved from the task of filtering the information received from the message hub, allowing the subscribing module to devote more resources to other operations.

Abstract: Objects placed on a flat surface are identified and localized by using a single view image. The single view image in the perspective projection is transformed to a normalized image in a pseudo plan to view to enhance detection of the bottom or top shapes of the objects. One or more geometric features are detected from the normalized image by processing the normalized image. The detected geometric features are analyzed to determine the identity and the location the objects on the flat surface.

Abstract: Multi-modal planning method and system that search a path for the most constrained mode first, and then expands the searches for path in a less constrained mode. By searching the path for the most constrained mode first, less resource are wasted on searching for paths that does not result in a feasible path in the most constrained mode. Multi-modal planning is performed by precomputing feasibility and utility of transition configurations of two adjacent modes. The feasibility is used to exclude non-feasible transition configurations in the most constrained mode from being sampled. The utility is used to bias sampling of the transition configuration so that transition configurations with higher utility are sampled with higher probability. Paths of configurations with higher utility and efficiency are obtained by biasing the sampling of the transition configurations.

Abstract: A system and method for object modeling. A compositing module embeds geometric shapes to model varying densities of an object. The compositing module represents each volume domain of constant density within a segment with a separate geometric shape. A mass set properties module in the system calculates estimates of mass set properties such as center of mass by first subtracting integrals related, for example, to the lungs from an integral related to the total chest cavity. A deforming module in the system provides a geometric shape that is deformable to more closely match data points from the object. The deforming module adjusts a B-spline solid having control points to fit an irregularly shaped object. Movement and location of the control points locally adjust the B-spline solid's surface. These variation can be used to determine mass set properties including mass, volume, center of mass, inertia tensors, principal moments, and the like.

Abstract: An extensible task engine framework for humanoid robots. Robot instructions are stored as tasks and skills. Tasks are designed so that they can be executed by a variety of robots with differing configurations. A task can refer to zero or more skills. A skill can be designed for a particular configuration of robot. A task can be transferred from robot to robot. When executed on a particular robot, the task makes calls to one or more skills that can take advantage of the capabilities of that robot.

Abstract: A general joint component framework that is capable of exhibiting complex behaviors of joints in articulated figures is provided. A network of joint components is used to model the kinematics of a joint. A joint builder can specify parameters for each of the joint components and join the joint components to form a joint set function that captures the biomechanical dependencies between the components. The joint function has fewer inputs than the total number of possible articulations yielding both simple control and biomechanically accurate joint movement.

Abstract: A system, a method and a computer readable medium are provided for simulating a combined musculoskeletal and augmentation device system. The dynamics model of the combined musculoskeletal and augmentation device system receives computed torques at the joints as inputs and delivers simulated kinematic data of the segments as outputs. The augmentation device controller for control of the augmentation device, receives the simulated kinematic data as inputs and delivers assist torques as outputs. The inverse dynamics model for the musculoskeletal and augmentation device system, receives the simulated kinematic data, desired kinematic data of the segments and the assit torques as inputs and delivers the computed net joint torque and muscle torque. The muscle force and muscle capacity module for checking and adjusting the computed torques, receives the computed torques as inputs and delivers computed torques as outputs after the checking and adjustment.

Abstract: An extensible task engine framework for humanoid robots. Robot instructions are stored as tasks and skills. Tasks are designed so that they can be executed by a variety of robots with differing configurations. A task can refer to zero or more skills. A skill can be designed for a particular configuration of robot. A task can be transferred from robot to robot. When executed on a particular robot, the task makes calls to one or more skills that can take advantage of the capabilities of that robot.

Abstract: A system and method for animating a character with activation-driven muscle deformation. External loads can be estimated through an iterative joint torque estimation process, and the external loads reflected in a physical model. Kinematic motion and the physical model reflecting external loads can be used to estimate joint torques. Muscle activations can be determined from the joint torques, and a character can be animated with muscle deformation responsive to the muscle activations.

Abstract: Systems and methods are presented that enable robot commands to be determined based on the pose of an object. A method is described for determining the orientation and pose of an object using indistinguishable points. The resolution of the pose detection is based on the robot'command vocabulary. A system is described for controlling a robot by pose detection using unlabelled points.