Abstract: An example method includes determining objects and actions associated with the objects for completing a task to be executed by a robotic system, where each action is associated with trajectory. The method further includes determining a pose for each person in an environment associated with the robotic system, predicting a trajectory for each person based on the determined pose associated with the respective person and the actions and trajectories associated with the actions, and adjusting trajectories for one or more of the actions to be performed by the robotic system based on the predicted trajectories for each person.

Abstract: In one embodiment, a method includes accessing a trajectory plan for a task to be executed by a robotic system, determining actions to constrain the trajectory plan based on information associated with an environment associated with the robotic system, wherein pose-based waypoints and joint positions of the robotic system would be constrained by the actions, determining joint-based waypoints for the trajectory plan based on the pose-based waypoints, and executing the task based on the joint-based waypoints for the trajectory plan.

Abstract: A method includes accessing RGB and depth image data representing a scene that includes at least a portion of a robotic limb. Using this data, a computing system may segment the image data to isolate and identify at least a portion of the robotic limb within the scene. The computing system can determine a current pose of the robotic limb within the scene based on the image data, joint data, or a 3D virtual model of the robotic limb. The computing system may then determine a desired goal pose, which may be based on the image data or the 3D virtual model. Based on the determined goal pose, the computing device determines the difference between the current pose and the goal pose of the robotic limb, and using this difference, provides a pose adjustment that for the robotic limb.

Abstract: A method includes accessing RGB and depth image data representing a scene that includes at least a portion of a robotic limb. Using this data, a computing system may segment the image data to isolate and identify at least a portion of the robotic limb within the scene. The computing system can determine a current pose of the robotic limb within the scene based on the image data, joint data, or a 3D virtual model of the robotic limb. The computing system may then determine a desired goal pose, which may be based on the image data or the 3D virtual model. Based on the determined goal pose, the computing device determines the difference between the current pose and the goal pose of the robotic limb, and using this difference, provides a pose adjustment that for the robotic limb.

Abstract: In one embodiment, a method includes accessing a trajectory plan for a task to be executed by a robotic system, determining actions to constrain the trajectory plan based on information associated with an environment associated with the robotic system, wherein pose-based waypoints and joint positions of the robotic system would be constrained by the actions, determining joint-based waypoints for the trajectory plan based on the pose-based waypoints, and executing the task based on the joint-based waypoints for the trajectory plan.

Abstract: In one embodiment, a method includes generating a trajectory plan to complete a task to be executed by a robotic system, identifying objects in the environment required for completing the task, determining attributes for each of the identified objects, determining trajectory-parameters for the trajectory plan based on the determined attributes for each identified object and operational conditions in an environment associated with the robotic system, and executing the task based on the determined trajectory-parameters for the trajectory plan.

Abstract: In one embodiment, a method includes determining objects and actions associated with the objects for completing a task to be executed by a robotic system, wherein each action is associated with trajectory, determining a pose for each person in an environment associated with the robotic system, predicting a trajectory for each person based on the determined pose associated with the respective person and the actions and trajectories associated with the actions, and adjusting trajectories for one or more of the actions to be performed by the robotic system based on the predicted trajectories for each person.

Abstract: A method includes accessing RGB and depth image data representing a scene that includes at least a portion of a robotic limb. Using this data, a computing system may segment the image data to isolate and identify at least a portion of the robotic limb within the scene. The computing system can determine a current pose of the robotic limb within the scene based on the image data, joint data, or a 3D virtual model of the robotic limb. The computing system may then determine a desired goal pose, which may be based on the image data or the 3D virtual model. Based on the determined goal pose, the computing device determines the difference between the current pose and the goal pose of the robotic limb, and using this difference, provides a pose adjustment that for the robotic limb.

Abstract: Providing assistance using a device can include determining, using a processor, an application that is currently executing and displayed by the device and determining, using the processor, information that accelerates user interaction with the application. The information can be displayed on the device as application-specific assistance.

Abstract: A system and method for a hybrid multimode input device includes a multimode input device capable of a two-dimensional (2D) mode of operation and a three-dimensional (3D) mode of operation. The device includes a two-dimensional (2D) position sensor, a three-dimensional (3D) position sensor, and a processor. The processor is configured to enter a 3D input mode when the processor detects that the input device is in a first orientation, and enter a 2D input mode when the processor detects that the input device is in a second orientation. The 2D input mode is configured to provide 2D position data to a connected computer system, and wherein the 3D input mode is configured to provide 3D position data to the connected computer system.

Abstract: The present disclosure includes methods and devices for generation and control of virtual displays in a virtual environment. A method includes receiving a trigger input from a user of a computing device, the computing device including a physical display that displays first content, initiating at least one virtual display based on the trigger input from the user, providing the at least one virtual display in association with the physical display, and displaying second content via the at least one virtual display in association with the first content displayed via the physical display.

Abstract: A method includes accessing RGB and depth image data representing a scene that includes at least a portion of a robotic limb. Using this data, a computing system may segment the image data to isolate and identify at least a portion of the robotic limb within the scene. The computing system can determine a current pose of the robotic limb within the scene based on the image data, joint data, or a 3D virtual model of the robotic limb. The computing system may then determine a desired goal pose, which may be based on the image data or the 3D virtual model. Based on the determined goal pose, the computing device determines the difference between the current pose and the goal pose of the robotic limb, and using this difference, provides a pose adjustment that for the robotic limb.

Abstract: A system and method for a hybrid multimode input device includes a multimode input device capable of a two-dimensional (2D) mode of operation and a three-dimensional (3D) mode of operation. The device includes a two-dimensional (2D) position sensor, a three-dimensional (3D) position sensor, and a processor. The processor is configured to enter a 3D input mode when the processor detects that the input device is in a first orientation, and enter a 2D input mode when the processor detects that the input device is in a second orientation. The 2D input mode is configured to provide 2D position data to a connected computer system, and wherein the 3D input mode is configured to provide 3D position data to the connected computer system.

Abstract: The present disclosure includes methods and devices for generation and control of virtual displays in a virtual environment. A method includes receiving a trigger input from a user of a computing device, the computing device including a physical display that displays first content, initiating at least one virtual display based on the trigger input from the user, providing the at least one virtual display in association with the physical display, and displaying second content via the at least one virtual display in association with the first content displayed via the physical display.

Abstract: Providing assistance using a device can include determining, using a processor, an application that is currently executing and displayed by the device and determining, using the processor, information that accelerates user interaction with the application. The information can be displayed on the device as application-specific assistance.