Abstract: A system and method of operating a mobile robot to perform tasks includes representing a task in an Object-Oriented Partially Observable Markov Decision Process model having at least one belief pertaining to a state and at least one observation space within an environment, wherein the state is represented in terms of classes and objects and each object has at least one attribute and a semantic label. The method further includes receiving a language command identifying a target object and a location corresponding to the target object, updating the belief associated with the target object based on the language command, driving the mobile robot to the observation space identified in the updated belief, searching the updated observation space for each instance of the target object, and providing notification upon completing the task. In an embodiment, the task is a multi-object search task.

Abstract: A method includes, as a robot encounters an object, creating a probabilistic object model to identify, localize, and manipulate the object, the probabilistic object model using light fields to enable efficient inference for object detection and localization while incorporating information from every pixel observed from across multiple camera locations.

Abstract: A method includes providing a robot, providing an image of drawn handwritten characters to the robot, enabling the robot to capture a bitmapped image of the image of drawn handwritten characters, enabling the robot to infer a plan to replicate the image with a writing utensil, and enabling the robot to reproduce the image.

Abstract: A method of representing a space of handwriting stroke styles includes representing writer-, character- and writer-character-level style variations within a recurrent neural network (RNN) model using decoupled style descriptors (DSD) that model the style variations such that character style variations depend on writer style.

Abstract: A method includes providing a robot, providing an image of drawn handwritten characters to the robot, enabling the robot to capture a bitmapped image of the image of drawn handwritten characters, enabling the robot to infer a plan to replicate the image with a writing utensil, and enabling the robot to reproduce the image.

Abstract: A system and method of operating a mobile robot to perform tasks includes representing a task in an Object-Oriented Partially Observable Markov Decision Process model having at least one belief pertaining to a state and at least one observation space within an environment, wherein the state is represented in terms of classes and objects and each object has at least one attribute and a semantic label. The method further includes receiving a language command identifying a target object and a location corresponding to the target object, updating the belief associated with the target object based on the language command, driving the mobile robot to the observation space identified in the updated belief, searching the updated observation space for each instance of the target object, and providing notification upon completing the task. In an embodiment, the task is a multi-object search task.

Abstract: A system includes a robot having a module that includes a function for mapping natural language commands of varying complexities to reward functions at different levels of abstraction within a hierarchical planning framework, the function including using a deep neural network language model that learns how to map the natural language commands to reward functions at an appropriate level of the hierarchical planning framework.

Abstract: A method includes enabling a robot to learn a mapping between English language commands and Linear Temporal Logic (LTL) expressions, wherein neural sequence-to-sequence learning models are employed to infer a LTL sequence corresponding to a given natural language command.

Abstract: A method includes, as a robot encounters an object, creating a probabilistic object model to identify, localize, and manipulate the object, the probabilistic object model using light fields to enable efficient inference for object detection and localization while incorporating information from every pixel observed from across multiple camera locations.

Abstract: A robot includes a gripping member configured to move and pick up the object, a camera affixed to the gripping member such that movement of the gripping member causes movement of the camera, the camera configured to measure and store data related to intensity of light and direction of light rays within the environment, an image processing module configured to process the data to generate a probabilistic model defining a location of the object within the environment, and an operation module configured to move the gripping member to the location and pick up the object.

Abstract: A system includes a robot having a module that includes a function for mapping natural language commands of varying complexities to reward functions at different levels of abstraction within a hierarchical planning framework, the function including using a deep neural network language model that learns how to map the natural language commands to reward functions at an appropriate level of the hierarchical planning framework.

Abstract: A system includes a robot having a module that includes a function for mapping natural language commands of varying complexities to reward functions at different levels of abstraction within a hierarchical planning framework, the function including using a deep neural network language model that learns how to map the natural language commands to reward functions at an appropriate level of the hierarchical planning framework.

Abstract: A method includes enabling a robot to learn a mapping between English language commands and Linear Temporal Logic (LTL) expressions, wherein neural sequence-to-sequence learning models are employed to infer a LTL sequence corresponding to a given natural language command.

Abstract: A system includes a robot having a module that includes a function for mapping natural language commands of varying complexities to reward functions at different levels of abstraction within a hierarchical planning framework, the function including using a deep neural network language model that learns how to map the natural language commands to reward functions at an appropriate level of the hierarchical planning framework.

Abstract: A robot includes a gripping member configured to move and pick up the object, a camera affixed to the gripping member such that movement of the gripping member causes movement of the camera, the camera configured to measure and store data related to intensity of light and direction of light rays within the environment, an image processing module configured to process the data to generate a probabilistic model defining a location of the object within the environment, and an operation module configured to move the gripping member to the location and pick up the object.