Abstract: Systems and methods for point marking using virtual fiducial elements are disclosed. An example method includes placing a set of fiducial elements in a locale or on an object and capturing a set of calibration images using an imager. The set of fiducial elements is fully represented in the set of calibration images. The method also includes generating a three-dimensional geometric model of the set of fiducial elements using the set of calibration images. The method also includes capturing a run time image of the locale or object. The run time image does not include a selected fiducial element, from the set of fiducial elements, which was removed from a location in the locale or on the object prior to capturing the run time image. The method concludes with identifying the location relative to the run time image using the run time image and the three-dimensional geometric model.

Abstract: Trained networks configured to detect fiducial elements in encodings of images and associated methods are disclosed. One method includes instantiating a trained network with a set of internal weights which encode information regarding a class of fiducial elements, applying an encoding of an image to the trained network where the image includes a fiducial element from the class of fiducial elements, generating an output of the trained network based on the set of internal weights of the network and the encoding of the image, and providing a position for at least one fiducial element in the image based on the output. Methods of training such networks are also disclosed.

Abstract: Systems and methods are provided for specifying safety rules for robotic devices. A computing device can determine information about any actors present within a predetermined area of an environment. The computing device can determine a safety classification for the predetermined area based on the information. The safety classification can include: a low safety classification if the information indicates zero actors are present within the predetermined area, a medium safety classification if the information indicates any actors are present within the predetermined area all are of a predetermined first type, and a high safety classification if the information indicates at least one actor present within the predetermined area is of a predetermined second type. After determining the safety classification for the predetermined area, the computing device can provide a safety rule for operating within the predetermined area to a robotic device operating in the environment.

Abstract: A control system may perform functions including (i) storing data indicating an association between an optical identifier and a first robot, (ii) sending, to the first robot, data encoding the optical identifier for display by the first robot, and (iii) after sending the data encoding the optical identifier, sending, to a second robot, the data indicating the association between the optical identifier and the first robot. In some examples, the first robot may receive, from the control system, data encoding a second optical identifier of the first robot so that the first robot may display the second optical identifier instead of the first optical identifier. In some examples, a first robot may capture an image of an indication of a priority status of a second robot and perform an action based on comparing a first priority status of the first robot to the second priority status of the second robot.

Abstract: This disclosure is directed to a hardware system for inverse graphics capture. An inverse graphics capture system (IGCS) captures data regarding a physical space that can be used to generate a photorealistic graphical model of that physical space. In certain approaches, the system includes hardware and accompanying software used to create a photorealistic six degree of freedom (6DOF) graphical model of the physical space. In certain approaches, the system includes hardware and accompanying software used for projection mapping onto the physical space. In certain approaches, the model produced by the IGCS is built using data regarding the geometry, lighting, surfaces, and environment of the physical space. In certain approaches, the model produced by the IGCS is both photorealistic and fully modifiable.

Abstract: Example embodiments may relate to methods and systems for selecting a grasp point on an object. In particular, a robotic manipulator may identify characteristics of a physical object within a physical environment. Based on the identified characteristics, the robotic manipulator may determine potential grasp points on the physical object corresponding to points at which a gripper attached to the robotic manipulator is operable to grip the physical object. Subsequently, the robotic manipulator may determine a motion path for the gripper to follow in order to move the physical object to a drop-off location for the physical object and then select a grasp point, from the potential grasp points, based on the determined motion path. After selecting the grasp point, the robotic manipulator may grip the physical object at the selected grasp point with the gripper and move the physical object through the determined motion path to the drop-off location.

Abstract: Example embodiments may relate to methods and systems for selecting a grasp point on an object. In particular, a robotic manipulator may identify characteristics of a physical object within a physical environment. Based on the identified characteristics, the robotic manipulator may determine potential grasp points on the physical object corresponding to points at which a gripper attached to the robotic manipulator is operable to grip the physical object. Subsequently, the robotic manipulator may determine a motion path for the gripper to follow in order to move the physical object to a drop-off location for the physical object and then select a grasp point, from the potential grasp points, based on the determined motion path. After selecting the grasp point, the robotic manipulator may grip the physical object at the selected grasp point with the gripper and move the physical object through the determined motion path to the drop-off location.

Abstract: Example implementations relate to determining depth information using stereo sensor data. An example system may include at least one projector coupled to a robotic manipulator and configured to project a texture pattern onto an environment. The system may further include a displacer coupled to the at least one texture projector and configured to repeatedly change a position of the texture pattern within the environment. The system may also include at least two optical sensors configured to capture stereo sensor data for the environment. And the system may include a computing device configured to determine, using the stereo sensor data, an output including a virtual representation of the environment.

Abstract: Systems and methods are provided for specifying safety rules for robotic devices. A computing device can determine information about any actors present within a predetermined area of an environment. The computing device can determine a safety classification for the predetermined area based on the information. The safety classification can include: a low safety classification if the information indicates zero actors are present within the predetermined area, a medium safety classification if the information indicates any actors are present within the predetermined area all are of a predetermined first type, and a high safety classification if the information indicates at least one actor present within the predetermined area is of a predetermined second type. After determining the safety classification for the predetermined area, the computing device can provide a safety rule for operating within the predetermined area to a robotic device operating in the environment.

Abstract: This disclosure is directed to a hardware system for inverse graphics capture. An inverse graphics capture system (IGCS) captures data regarding a physical space that can be used to generate a photorealistic graphical model of that physical space. In certain approaches, the system includes hardware and accompanying software used to create a photorealistic six degree of freedom (6DOF) graphical model of the physical space. In certain approaches, the system includes hardware and accompanying software used for projection mapping onto the physical space. In certain approaches, the model produced by the IGCS is built using data regarding the geometry, lighting, surfaces, and environment of the physical space. In certain approaches, the model produced by the IGCS is both photorealistic and fully modifiable.

Abstract: An example method involves receiving, from at least one camera located in an environment, a plurality of images captured during a first time interval. The method also involves selecting one or more of the plurality of images having a movable platform supporting one or more objects. The method further involves generating a three-dimensional model of the movable platform supporting the one or more objects. The method yet further involves updating the three-dimensional model based on one or more images captured during a second time interval. The method still further involves presenting the three-dimensional model via a display of a user interface, and providing an option to view a history of the three-dimensional model such that the three-dimensional model remains in a fixed position on the display during a viewing of the history.

Abstract: Systems and methods are provided for specifying safety rules for robotic devices. A computing device can determine information about any actors present within a predetermined area of an environment. The computing device can determine a safety classification for the predetermined area based on the information. The safety classification can include: a low safety classification if the information indicates zero actors are present within the predetermined area, a medium safety classification if the information indicates any actors are present within the predetermined area all are of a predetermined first type, and a high safety classification if the information indicates at least one actor present within the predetermined area is of a predetermined second type. After determining the safety classification for the predetermined area, the computing device can provide a safety rule for operating within the predetermined area to a robotic device operating in the environment.

Abstract: Example embodiments may relate to methods and systems for selecting a grasp point on an object. In particular, a robotic manipulator may identify characteristics of a physical object within a physical environment. Based on the identified characteristics, the robotic manipulator may determine potential grasp points on the physical object corresponding to points at which a gripper attached to the robotic manipulator is operable to grip the physical object. Subsequently, the robotic manipulator may determine a motion path for the gripper to follow in order to move the physical object to a drop-off location for the physical object and then select a grasp point, from the potential grasp points, based on the determined motion path. After selecting the grasp point, the robotic manipulator may grip the physical object at the selected grasp point with the gripper and move the physical object through the determined motion path to the drop-off location.

Abstract: Methods and systems for providing landmarks to facilitate robot localization and visual odometry are provided herein. At least one area of a physical environment in which a robotic device resides may be determined to include surfaces that lack sufficient discernable features to determine a location of the at least one area. Instructions may responsively be provided to the robotic device for the robotic device to provide a material in respective patterns onto one or more surfaces of the at least one area. Instructions can responsively be provided for the robotic device to provide the material in respective textures as well. The respective patterns or textures may include sufficient discernable features to determine a location of the at least one area, and the material may remain on the one or more surfaces for a predetermined period of time.

Abstract: Example implementations may relate to accurate pallet insertion. An example system may include a forklifted configured with a base and a tine coupled to the base. The tine may include a first proximity sensor positioned on a first side of the tine and a second proximity sensor positioned on a second side of the tine. The system may also include a computing system configured to receive, from the first sensor and the second sensor, sensor data indicative of proximity of the tine relative to opposing surfaces of a pocket of a pallet and navigate the forklift relative to the pallet based on the sensor data.

Abstract: This disclosure is directed to a hardware system for inverse graphics capture. An inverse graphics capture system (IGCS) captures data regarding a physical space that can be used to generate a photorealistic graphical model of that physical space. In certain approaches, the system includes hardware and accompanying software used to create a photorealistic six degree of freedom (6DOF) graphical model of the physical space. In certain approaches, the system includes hardware and accompanying software used for projection mapping onto the physical space. In certain approaches, the model produced by the IGCS is built using data regarding the geometry, lighting, surfaces, and environment of the physical space. In certain approaches, the model produced by the IGCS is both photorealistic and fully modifiable.

Abstract: Methods and systems for using projected patterns to facilitate mapping of an environment are provided herein. A computing system may cause fixedly-posed projectors to each provide, onto a respective area of the environment, a predetermined respective distinct pattern. The system may determine respective poses of the projectors, and further determine a map of the environment that identifies, for each distinct pattern, respective locations on one or more surfaces in the environment on which the distinct pattern is detectable. Based on sensor data the system may identify a portion of a particular distinct pattern in the environment. The system may use the map and the respective pose of a particular projector that is providing the particular pattern to make a determination that the portion is located at a new, different location compared to the map. The system may then transmit an output signal indicating the determination.

Abstract: Example embodiments may relate to methods and systems for selecting a grasp point on an object. In particular, a robotic manipulator may identify characteristics of a physical object within a physical environment. Based on the identified characteristics, the robotic manipulator may determine potential grasp points on the physical object corresponding to points at which a gripper attached to the robotic manipulator is operable to grip the physical object. Subsequently, the robotic manipulator may determine a motion path for the gripper to follow in order to move the physical object to a drop-off location for the physical object and then select a grasp point, from the potential grasp points, based on the determined motion path. After selecting the grasp point, the robotic manipulator may grip the physical object at the selected grasp point with the gripper and move the physical object through the determined motion path to the drop-off location.

Abstract: Example implementations may relate to accurate pallet insertion. An example system may include a forklifted configured with a base and a tine coupled to the base. The tine may include a first proximity sensor positioned on a first side of the tine and a second proximity sensor positioned on a second side of the tine. The system may also include a computing system configured to receive, from the first sensor and the second sensor, sensor data indicative of proximity of the tine relative to opposing surfaces of a pocket of a pallet and navigate the forklift relative to the pallet based on the sensor data.

Abstract: An example method includes receiving, by a mobile robotic device, power from a battery of a first battery pack in order to operate within an environment. The method further includes establishing a first data channel between the mobile robotic device and the first battery pack. The method also includes using the first data channel to transfer sensor data acquired by the mobile robotic device during operation to a local data storage component of the first battery pack. The method additionally includes navigating, by the mobile robotic device, to a battery exchange station to transfer the first battery pack containing the battery and the local data storage component with the sensor data to the battery exchange station. The method further includes after transferring the first battery pack to the battery exchange station, receiving a second battery pack from the battery exchange station to continue operation within the environment.