Abstract: Methods and systems for determining a status of a component of a robotic device are provided. An example method includes triggering an action of a component of a robotic device, and responsively receiving information associated with the action of the component from a sensor. The method further includes a computing system having a processor and a memory comparing the information with calibration data and determining a status of the component based on the comparison. In some examples, the calibration data may include information derived from data received from a pool of one or more robotic devices utilizing same or similar components as the component. The determined status may include information associated with a performance of the component with respect to performances of same or similar components of the pool of robotic devices. In one example, the robotic device may self-calibrate the component based on the status.

Abstract: Methods and systems for interacting with multiple three-dimensional (3D) object data models are provided. An example method may involve providing to a display device for display a first 3D object data model and a second 3D object data model. Information associated with a modification to the first 3D object data model may be received. Based on the received information, a same change may be applied to the first 3D object data model and applied to the second 3D object data model to obtain a first modified 3D object data model and a second modified 3D object data model. According to the method, the first modified 3D object data model and the second modified 3D object data model may be provided to the display device for substantially simultaneous display.

Abstract: Methods and systems for encoding and compressing 3D object data models are provided. An example method may involve receiving 3D mesh data for an object that includes geometry coordinates for a surface of the object. Additionally, material properties may be associated with the geometry coordinates. The method may also include identifying multiple portions of the mesh data based on the material properties associated with the geometry coordinates. For example, a given group of adjacent geometry coordinates having common material properties may be identified as a given portion. For at least some of the identified portions of the mesh data, the method may further include encoding information related to an identified portion of the mesh data and compressing the encoded information into a file of compressed geometric data.

Abstract: Methods and systems for superimposing virtual views of 3D objects with live images are provided. An example method may include determining 3D geometry information associated with an environment using one or more sensors of a mobile device, and recognizing 3D objects in the environment based on a database of 3D objects and the 3D geometry information. The method may also include determining a field of view of the environment based on a position of the mobile device in the environment and rendering an image of one or more of the 3D objects that are within the field of view. The rendered image may have a given perspective that is based on a viewpoint position from which the mobile device is viewed. The rendered image may also be combined with a live image of the field of view that is presented by the mobile device.

Abstract: Robots, computer program products, and methods for trajectory plan optimization are disclosed. In one embodiment, a method of controlling a robot having a first manipulator and a second manipulator includes receiving a trajectory plan including a plurality of sequential motion segments. The method further includes determining a moveable motion segment, and shifting the moveable motion segment and motion segments subsequent to the moveable motion segment backward in time to a shifted time such that one or more unshifted segments of the trajectory plan occur at a same time as one or more shifted segment segments. The method may further include controlling the robot according to the optimized trajectory plan such that one or more components of the first manipulator are moved concurrently with one or more components of the second manipulator.

Abstract: Methods and systems for interacting with multiple three-dimensional (3D) object data models are provided. An example method may involve providing to a display device for display a first 3D object data model and a second 3D object data model. Information associated with a modification to the first 3D object data model may be received. Based on the received information, a same change may be applied to the first 3D object data model and applied to the second 3D object data model to obtain a first modified 3D object data model and a second modified 3D object data model. According to the method, the first modified 3D object data model and the second modified 3D object data model may be provided to the display device for substantially simultaneous display.

Abstract: Methods and systems for encoding and compressing 3D object data models are provided. An example method may involve receiving 3D mesh data for an object that includes geometry coordinates for a surface of the object. Additionally, material properties may be associated with the geometry coordinates. The method may also include identifying multiple portions of the mesh data based on the material properties associated with the geometry coordinates. For example, a given group of adjacent geometry coordinates having common material properties may be identified as a given portion. For at least some of the identified portions of the mesh data, the method may further include encoding information related to an identified portion of the mesh data and compressing the encoded information into a file of compressed geometric data.

Abstract: Methods and systems for writing, interpreting, and translating three-dimensional (3D) scenes are provided. An example method may involve accessing data associated with a three-dimensional (3D) scene that includes one or more objects of the 3D scene and one or more rendering effects for the one or more objects. Requests for assets and instructions associated with rendering the one or more objects based on the data associated with the 3D scene may be determined and sent to a server. Additionally, the method may include receiving from the server assets and instructions that facilitate rendering the one or more objects based on the one or more rendering effects. According to the method, the one or more objects of the 3D scene may be rendered based on the received instructions and the received assets.

Abstract: Methods and systems for interacting with multiple three-dimensional (3D) object data models are provided. An example method may involve providing to a display device for display a first 3D object data model and a second 3D object data model. Information associated with a modification to the first 3D object data model may be received. Based on the received information, a same change may be applied to the first 3D object data model and applied to the second 3D object data model to obtain a first modified 3D object data model and a second modified 3D object data model. According to the method, the first modified 3D object data model and the second modified 3D object data model may be provided to the display device for substantially simultaneous display.

Abstract: Methods and systems for determining three-dimensional (3D) object data models of an object based on movement of the object. A processor may receive first scan data and second scan data of an object within in a space. The first scan data may comprise a first position of the object in the space and a first mesh image defining a first view of a shape of the object at the first position. The second scan data may comprise movement information defining a movement of the object from the first position to a second position within the space and a second mesh image defining a second view of the shape of the object at the second position. The first scan data and the second scan data may be aligned to generate a 3D object data model based on the first mesh image, the movement information, and the second mesh image.

Abstract: A method and system for video encoding assets for swivel/360-degree spinners is disclosed. Still images of a 3D object from different perspectives about the 3D object may be stacked and then video encoded to generate video frames of the object from the different perspectives. The video-encoded assets may be stored on a server or other network-connected device, and later retrieved by a connected client device for display processing by a swivel/360-degree spinner on the client device. The swivel/360-degree spinner may utilize native video processing capabilities of the client device and/or of a browser running on the client device to display video motion of the object moving through different angular orientations in response to movement of an interactive cursor.

Abstract: The present application discloses shared robot knowledge bases for use with cloud computing systems. In one embodiment, the cloud computing system collects data from a robot about an object the robot has encountered in its environment, and stores the received data in the shared robot knowledge base. In another embodiment, the cloud computing system sends instructions for interacting with an object to a robot, receives feedback from the robot based on its interaction with the object, and updates data in the shared robot knowledge base based on the feedback. In yet another embodiment, the cloud computing system sends instructions to a robot for executing an application based on information stored in the shared robot knowledge base. In the disclosed embodiments, information in the shared robot knowledge bases is updated based on robot experiences so that any particular robot may benefit from prior experiences of other robots.

Abstract: Methods and systems for determining three-dimensional (3D) object data models of an object based on movement of the object. A processor may receive first scan data and second scan data of an object within in a space. The first scan data may comprise a first position of the object in the space and a first mesh image defining a first view of a shape of the object at the first position. The second scan data may comprise movement information defining a movement of the object from the first position to a second position within the space and a second mesh image defining a second view of the shape of the object at the second position. The first scan data and the second scan data may be aligned to generate a 3D object data model based on the first mesh image, the movement information, and the second mesh image.

Abstract: Methods and systems are provided for determining and transmitting applicable lighting information, applicable viewing perspective, and a 3D model for an object in response to a search query. An example method includes receiving, at a server, a search query regarding an object. A 3D model for the object is determined. The 3D model includes three-dimensional shape information about the object. The method also includes determining, based on a plurality of stored images of the object, at least one applicable light field and at least one applicable viewing perspective. A search query result is transmitted from the server. The search query result may include the 3D model, the applicable light field(s), and the applicable viewing perspective(s). A server and a non-transitory computer readable medium are also disclosed that could perform a similar method.

Abstract: Methods and systems for material refinement for portions of a three-dimensional (3D) object data model are provided. An example method may include rendering a portion of a 3D object data model, and determining a first appearance metric between an appearance of the portion in the rendered view and a two-dimensional (2D) image. For one or more iterations, a modification to material properties associated with the portion may be determined based on the first appearance metric, and another view of the portion of the 3D object data model may be rendered. Also for the one or more iterations, another appearance metric between and an appearance of the portion in the rendered another view and the 2D image may be determined. Additionally, modified material properties for the portion that are associated with a minimum appearance metric of the one or more iterations may be stored for the 3D object data model.

Abstract: Methods and systems for interacting with multiple three-dimensional (3D) object data models are provided. An example method may involve receiving an annotated template of a first three-dimensional (3D) object data model. The annotated template may be associated with a given category of objects and may include one or more annotations to one or more aspects of an object described by the first 3D object data model. The method may also include determining matching correspondences between the first 3D object data model and a second 3D object data model that is a given object in the given category. For the one or more matching correspondences, an annotation that is associated with a correspondence point of the first 3D object data model may be applied as a call-out to a matching correspondence point of the second 3D object data model.

Abstract: Methods and computer program products for generating robot grasp patterns are disclosed. In one embodiment, a method for generating robot grasp patterns includes generating a plurality of approach rays associated with a target object. Each approach ray of the plurality of approach rays extends perpendicularly from a surface of the target object. The method further includes generating at least one grasp pattern for each approach ray to generate a grasp pattern set of the target object, calculating a grasp quality score for each individual grasp pattern of the grasp pattern set, and comparing the grasp quality score of each individual grasp pattern with a grasp quality threshold. The method further includes selecting individual grasp patterns of the grasp pattern set having a grasp quality score that is greater than the grasp quality threshold, and providing the selected individual grasp patterns to the robot for on-line manipulation of the target object.

Abstract: Methods and computer-program products for evaluating grasp patterns for use by a robot are disclosed. In one embodiment, a method of evaluating grasp patterns includes selecting an individual grasp pattern from a grasp pattern set, establishing a thumb-up vector, and simulating the motion of the manipulator and the end effector according to the selected individual grasp pattern, wherein each individual grasp pattern of the grasp pattern set corresponds to motion for manipulating a target object. The method further includes evaluating a direction of the thumb-up vector during at least a portion of the simulated motion of the manipulator and the end effector, and excluding the selected individual grasp pattern from use by the robot if the direction of the thumb-up vector during the simulated motion is outside of one or more predetermined thresholds. Robots utilizing the methods and computer-program products for evaluating grasp patterns are also disclosed.

Abstract: Robots, computer program products, and methods for trajectory plan optimization are disclosed. In one embodiment, a method of controlling a robot having a first manipulator and a second manipulator includes receiving a trajectory plan including a plurality of sequential motion segments. The method further includes determining a moveable motion segment, and shifting the moveable motion segment and motion segments subsequent to the moveable motion segment backward in time to a shifted time such that one or more unshifted segments of the trajectory plan occur at a same time as one or more shifted segment segments. The method may further include controlling the robot according to the optimized trajectory plan such that one or more components of the first manipulator are moved concurrently with one or more components of the second manipulator.

Abstract: Methods and systems for prioritization of display of portions of three-dimensional (3D) object data models are described. An example method may include receiving information associated with a plurality of renderings of a 3D model. The received information may identify portions of interest of the 3D model which are inspected while the 3D model is rendered on a display. The method may also include determining frequencies with which the portions of interest are inspected based on the received information, and determining priority values for the portions of interest based on the frequencies with which the portions of interest are inspected. According to the method, a statistical model for the 3D model may be determined via a processor based on the priority values for the portions of interest.