Abstract: An initial environment navigation model for a physical environment may be determined based on sensor data collected from a mobile enrollment device. The sensor data may include data collected from a first one or more cameras at the mobile enrollment device. The initial environment navigation model may be sent to a robot via a communication interface. The robot may be instructed to autonomously navigate the physical environment based on the initial environment navigation model and additional sensor data collected by the robot. An updated environment navigation model for the physical environment may be determined based on the initial environment navigation model and the additional sensor data.

Abstract: One or more simulated capture paths through a physical environment may be determined for a robot based on an environment navigation model of the physical environment. A plurality of simulated object parameter values may be determined for an object type. Simulated sensor data for a plurality of simulated instances of the object type may be determined based on the one or more simulated capture paths, the environment navigation model, and the simulated object parameter values. An object recognition model to recognize an object corresponding with the object type based on the simulated sensor data.

Abstract: A cleaning robot may determine a three-dimensional model of a physical environment based on data collected from one or more sensors. The cleaning robot may then identify a surface within the physical environment to clean. Having identified that surface, the robot may autonomously navigate to a location proximate to the surface, position an ultraviolet light source in proximity to the surface, and activate the ultraviolet light source for a period of time.

Abstract: This application describes systems, devices, computer readable media, and methods for the function and operation of robotic carts. A robotic cart may include a base component configured for the receipt of a payload, a battery unit, and a mobility apparatus. The robotic cart may include a handlebar component coupled with the base component. The handlebar unit may include a sensor unit configured to transmit a hand detection message when the handlebar unit is grasped by one or more hands and to transmit a force direction message indicating a two-dimensional direction associated with a directional force applied by one or more hands. The robotic cart may be configured to map the area around it and to autonomously move the robotic cart along a path to perform a task.

Abstract: Example implementations relate to punch-in transitions to smoothly transition operation of parts of a robotic system from control tracks to real-time input, such as to smoothly transition a control track rotating a knob at a handheld controller to real-time input received at the handheld controller. Similarly, example implementations relate to punch-out transitions to smoothly transition operation of parts of a robotic system from real-time input to control tracks, such as to smoothly transition from the real-time input received at a handheld controller to a control track rotating a knob at a handheld controller. In particular, an example system may include a handheld haptic controller, a control system, and a robotic component.

Abstract: Example implementations may relate to optimization of observer robot locations. In particular, a control system may detect an event that indicates desired relocation of observer robots within a worksite. Each such observer robot may have respective sensor(s) configured to provide information related to respective positions of a plurality of target objects within the worksite. Responsively, the control system may (i) determine observer robot locations within the worksite at which one or more of the respective sensors are each capable of providing information related to respective positions of one or more of the plurality of target objects and (ii) determine a respectively intended level of positional accuracy for at least two respective target objects. Based on the respectively intended levels of positional accuracy, the control system may select one or more of the observer robot locations and may direct one or more observer robots to relocate to the selected locations.

Abstract: In one aspect, a method is described. The method may include providing an end effector tool of a robotic device configured to perform a task on a work surface within a worksite coordinate frame. The method may further include providing first location data indicating a first location of the end effector tool with respect to the work surface, providing second location data indicating a second location of the end effector tool within the worksite coordinate frame, and providing third location data indicating a third location of the end effector tool within the worksite coordinate frame. The method may further include tracking the location of the end effector tool based on the first, second, and third location data, and, based on the tracked location of the tool, instructing the robotic device to manipulate the end effector tool to perform a task on the work surface.

Abstract: A cleaning robot may determine a three-dimensional model of a physical environment based on data collected from one or more sensors. The cleaning robot may then identify a surface within the physical environment to clean. Having identified that surface, the robot may autonomously navigate to a location proximate to the surface, position an ultraviolet light source in proximity to the surface, and activate the ultraviolet light source for a period of time.

Abstract: A robot may identify a human located proximate to the robot in a physical environment based on sensor data captured from one or more sensors on the robot. A trajectory of the human through space may be predicted. When the predicted trajectory of the human intersects with a current path of the robot, an updated path to a destination location in the environment may be determined so as to avoid a collision between the robot and the human along the predicted trajectory. The robot may then move along the determined path.

Abstract: A cleaning robot may include a chassis, a camera, a control unit, a mobility apparatus, and a movable lighting element. The control unit may execute computer programming instructions in a manner depending on information received from the camera. The mobility apparatus may be capable of causing the cleaning robot to move through a physical space.

Abstract: Example implementations may relate to a haptic hand-holdable controller. In particular, an example device may take the form of a haptic controller, which senses tactile information and provides force feedback for a more intuitive user experience. The force feedback may indicate a state of the device that is being controlled. An example haptic handheld controller may be utilized to manipulate data input to a robot, a tablet computer, and/or any other type of computing device. In an example embodiment, the haptic handheld controller may be such that the controller indicates to the user what manipulation of different types of data feels like, for example, by using operating modes for the haptic handheld controller where a motor varies feedback to the handheld controller.

Abstract: Robotic control systems and methods may include providing an end effector tool of a robotic device configured to perform a task on a work surface within a worksite coordinate frame. Unintended movement over time of the end effector tool with respect to the work surface and with respect to the worksite coordinate frame may be determined based on image data indicative of the work surface, first location data indicative of a first location of the end effector tool with respect to the worksite coordinate frame, and second location data indicative of a second location of the end effector tool with respect to the work surface. One or more control signals for the robotic device may be adjusted in order to counteract the unintended movements of the end effector tool with respect to the work surface and worksite coordinate frame.

Abstract: Methods, apparatus, systems, and computer-readable media are provided for visually annotating rendered multi-dimensional representations of robot environments. In various implementations, first and second respective states of an object that exist before and after the object is acted upon by one or more robots may be determined. In various implementations, a multi-dimensional representation of an environment in which the one or more robots operate may be rendered, e.g., as part of a graphical user interface. In various implementations, a graphical representation of the object in at least the first or second state and a visual annotation of a trajectory of the object between the first and second states may be rendered within the multi-dimensional representation of the environment.

Abstract: Robotic control systems and methods may include providing an end effector tool of a robotic device configured to perform a task on a work surface within a worksite coordinate frame. Unintended movement over time of the end effector tool with respect to the work surface and with respect to the worksite coordinate frame may be determined based on image data indicative of the work surface, first location data indicative of a first location of the end effector tool with respect to the worksite coordinate frame, and second location data indicative of a second location of the end effector tool with respect to the work surface. One or more control signals for the robotic device may be adjusted in order to counteract the unintended movements of the end effector tool with respect to the work surface and worksite coordinate frame.

Abstract: Example implementations may relate to optimization of observer robot locations. In particular, a control system may detect an event that indicates desired relocation of observer robots within a worksite. Each such observer robot may have respective sensor(s) configured to provide information related to respective positions of a plurality of target objects within the worksite. Responsively, the control system may (i) determine observer robot locations within the worksite at which one or more of the respective sensors are each capable of providing information related to respective positions of one or more of the plurality of target objects and (ii) determine a respectively intended level of positional accuracy for at least two respective target objects. Based on the respectively intended levels of positional accuracy, the control system may select one or more of the observer robot locations and may direct one or more observer robots to relocate to the selected locations.

Abstract: In one aspect, a method is described. The method may include providing an end effector tool of a robotic device configured to perform a task on a work surface within a worksite coordinate frame. The method may further include providing first location data indicating a first location of the end effector tool with respect to the work surface, providing second location data indicating a second location of the end effector tool within the worksite coordinate frame, and providing third location data indicating a third location of the end effector tool within the worksite coordinate frame. The method may further include tracking the location of the end effector tool based on the first, second, and third location data, and, based on the tracked location of the tool, instructing the robotic device to manipulate the end effector tool to perform a task on the work surface.

Abstract: Example implementations relate to punch-in transitions to smoothly transition operation of parts of a robotic system from control tracks to real-time input, such as to smoothly transition a control track rotating a knob at a handheld controller to real-time input received at the handheld controller. Similarly, example implementations relate to punch-out transitions to smoothly transition operation of parts of a robotic system from real-time input to control tracks, such as to smoothly transition from the real-time input received at a handheld controller to a control track rotating a knob at a handheld controller. In particular, an example system may include a handheld haptic controller, a control system, and a robotic component.

Abstract: Example implementations relate to punch-in transitions to smoothly transition operation of parts of a robotic system from control tracks to real-time input, such as to smoothly transition a control track rotating a knob at a handheld controller to real-time input received at the handheld controller. Similarly, example implementations relate to punch-out transitions to smoothly transition operation of parts of a robotic system from real-time input to control tracks, such as to smoothly transition from the real-time input received at a handheld controller to a control track rotating a knob at a handheld controller. In particular, an example system may include a handheld haptic controller, a control system, and a robotic component.

Abstract: Methods, apparatus, systems, and computer-readable media are provided for determining, based on a task to be performed by a robot and past behavior by robots while performing tasks similar to the task, a suggested task-level movement parameter for application to movement of the robot while performing the task; providing output indicative of the suggested task-level movement parameter; receiving input indicative of user selection of the suggested task-level movement parameter or a user-defined task-level movement parameter; determining, based on the received input, an actual task-level movement parameter to be applied to movement of the robot while performing the task; and identifying, based on the actual task-level movement parameter, a plurality of component-level movement parameters to be applied to a plurality of motion primitives implemented by one or more operational components of the robot to perform the task.

Abstract: Methods and apparatus that adapt programming and/or control of a robot. The robot has at least one attachment area that can removably couple with any one of a plurality of end effectors. The programming and/or control is adapted based on received parameters of an end effector attached (or to be attached) to the attachment area of the robot. Some implementations are directed to adapting graphical user interface output of a robot programming application based on one or more parameters of an end effector for a robot being programmed via the robot programming application. Some implementations are directed to adapting control of a robot based on one or more parameters of an end effector attached (or to be attached) to the robot.