Abstract: An example method includes determining objects and actions associated with the objects for completing a task to be executed by a robotic system, where each action is associated with trajectory. The method further includes determining a pose for each person in an environment associated with the robotic system, predicting a trajectory for each person based on the determined pose associated with the respective person and the actions and trajectories associated with the actions, and adjusting trajectories for one or more of the actions to be performed by the robotic system based on the predicted trajectories for each person.

Abstract: In one embodiment, a method includes accessing a trajectory plan for a task to be executed by a robotic system, determining actions to constrain the trajectory plan based on information associated with an environment associated with the robotic system, wherein pose-based waypoints and joint positions of the robotic system would be constrained by the actions, determining joint-based waypoints for the trajectory plan based on the pose-based waypoints, and executing the task based on the joint-based waypoints for the trajectory plan.

Abstract: In one embodiment, a method for monitoring a kitchen environment by a smart hood includes accessing sensor data indicating parameters of a cooking utensil detected by the smart hood apparatus from sensors of the smart hood apparatus, wherein the sensors comprise one or more of a camera, a thermal sensor, an audio sensor, or an air particle sensor, determining occupancy information associated with the cooking utensil based on the sensor data by algorithms, wherein the occupancy information indicates whether the cooking utensil is placed within an allowable deviation from a target location on a cooktop, determining a thermal state of an accessory associated with the cooking utensil based on the sensor data by the algorithms, and sending instructions for presenting the occupancy information associated with the cooking utensil and the thermal state of the accessory.

Abstract: In one embodiment, a method for monitoring for boiling fluid in a kitchen environment by a smart hood apparatus includes accessing sensor data indicating one or more parameters of a cooking utensil detected by the smart hood apparatus from sensors associated with the smart hood apparatus, wherein the sensors comprise one or more of a camera, a thermal sensor, or an audio sensor, and wherein the cooking utensil contains fluid, determining information associated with boiling events associated with the fluid in the cooking utensil based on the sensor data by algorithms, wherein the information comprises at least a boiling time required for the fluid to boil, and sending instructions for presenting a notification comprising the information associated with the boiling events.

Abstract: A method includes accessing RGB and depth image data representing a scene that includes at least a portion of a robotic limb. Using this data, a computing system may segment the image data to isolate and identify at least a portion of the robotic limb within the scene. The computing system can determine a current pose of the robotic limb within the scene based on the image data, joint data, or a 3D virtual model of the robotic limb. The computing system may then determine a desired goal pose, which may be based on the image data or the 3D virtual model. Based on the determined goal pose, the computing device determines the difference between the current pose and the goal pose of the robotic limb, and using this difference, provides a pose adjustment that for the robotic limb.

Abstract: A method includes accessing RGB and depth image data representing a scene that includes at least a portion of a robotic limb. Using this data, a computing system may segment the image data to isolate and identify at least a portion of the robotic limb within the scene. The computing system can determine a current pose of the robotic limb within the scene based on the image data, joint data, or a 3D virtual model of the robotic limb. The computing system may then determine a desired goal pose, which may be based on the image data or the 3D virtual model. Based on the determined goal pose, the computing device determines the difference between the current pose and the goal pose of the robotic limb, and using this difference, provides a pose adjustment that for the robotic limb.

Abstract: In one embodiment, a method includes accessing a trajectory plan for a task to be executed by a robotic system, determining actions to constrain the trajectory plan based on information associated with an environment associated with the robotic system, wherein pose-based waypoints and joint positions of the robotic system would be constrained by the actions, determining joint-based waypoints for the trajectory plan based on the pose-based waypoints, and executing the task based on the joint-based waypoints for the trajectory plan.

Abstract: In one embodiment, a method includes generating a trajectory plan to complete a task to be executed by a robotic system, identifying objects in the environment required for completing the task, determining attributes for each of the identified objects, determining trajectory-parameters for the trajectory plan based on the determined attributes for each identified object and operational conditions in an environment associated with the robotic system, and executing the task based on the determined trajectory-parameters for the trajectory plan.

Abstract: In one embodiment, a method includes determining objects and actions associated with the objects for completing a task to be executed by a robotic system, wherein each action is associated with trajectory, determining a pose for each person in an environment associated with the robotic system, predicting a trajectory for each person based on the determined pose associated with the respective person and the actions and trajectories associated with the actions, and adjusting trajectories for one or more of the actions to be performed by the robotic system based on the predicted trajectories for each person.

Abstract: A method includes accessing RGB and depth image data representing a scene that includes at least a portion of a robotic limb. Using this data, a computing system may segment the image data to isolate and identify at least a portion of the robotic limb within the scene. The computing system can determine a current pose of the robotic limb within the scene based on the image data, joint data, or a 3D virtual model of the robotic limb. The computing system may then determine a desired goal pose, which may be based on the image data or the 3D virtual model. Based on the determined goal pose, the computing device determines the difference between the current pose and the goal pose of the robotic limb, and using this difference, provides a pose adjustment that for the robotic limb.

Abstract: A system and method for a hybrid multimode input device includes a multimode input device capable of a two-dimensional (2D) mode of operation and a three-dimensional (3D) mode of operation. The device includes a two-dimensional (2D) position sensor, a three-dimensional (3D) position sensor, and a processor. The processor is configured to enter a 3D input mode when the processor detects that the input device is in a first orientation, and enter a 2D input mode when the processor detects that the input device is in a second orientation. The 2D input mode is configured to provide 2D position data to a connected computer system, and wherein the 3D input mode is configured to provide 3D position data to the connected computer system.

Abstract: The present disclosure includes methods and devices for generation and control of virtual displays in a virtual environment. A method includes receiving a trigger input from a user of a computing device, the computing device including a physical display that displays first content, initiating at least one virtual display based on the trigger input from the user, providing the at least one virtual display in association with the physical display, and displaying second content via the at least one virtual display in association with the first content displayed via the physical display.

Abstract: In one embodiment, a method includes transmitting, from a first transducer of an electronic device, a first audio signal to a surface near the electronic device. The first audio signal is generated based on a frequency sweep across a range of frequencies. The method also includes receiving, at a second transducer of the electronic device, a second audio signal that is at least partly reflected off the surface. The method then determines an attribute of the surface based on the received second audio signal.

Abstract: A method includes accessing RGB and depth image data representing a scene that includes at least a portion of a robotic limb. Using this data, a computing system may segment the image data to isolate and identify at least a portion of the robotic limb within the scene. The computing system can determine a current pose of the robotic limb within the scene based on the image data, joint data, or a 3D virtual model of the robotic limb. The computing system may then determine a desired goal pose, which may be based on the image data or the 3D virtual model. Based on the determined goal pose, the computing device determines the difference between the current pose and the goal pose of the robotic limb, and using this difference, provides a pose adjustment that for the robotic limb.

Abstract: A system and method for a hybrid multimode input device includes a multimode input device capable of a two-dimensional (2D) mode of operation and a three-dimensional (3D) mode of operation. The device includes a two-dimensional (2D) position sensor, a three-dimensional (3D) position sensor, and a processor. The processor is configured to enter a 3D input mode when the processor detects that the input device is in a first orientation, and enter a 2D input mode when the processor detects that the input device is in a second orientation. The 2D input mode is configured to provide 2D position data to a connected computer system, and wherein the 3D input mode is configured to provide 3D position data to the connected computer system.

Abstract: The present disclosure includes methods and devices for generation and control of virtual displays in a virtual environment. A method includes receiving a trigger input from a user of a computing device, the computing device including a physical display that displays first content, initiating at least one virtual display based on the trigger input from the user, providing the at least one virtual display in association with the physical display, and displaying second content via the at least one virtual display in association with the first content displayed via the physical display.

Abstract: In one embodiment, an apparatus includes an electrode that is coupled to a body of a user and is configured to receive a signal from the body. The received signal is based on an electromagnetic interference signal generated by an object that is external to the apparatus. The apparatus further includes one or more processors coupled to the electrode. The processors are configured to detect, based on the signal received by the electrode, one or more of: an interaction between the user and the object, an identity of the object, or a context surrounding the apparatus.

Abstract: In one embodiment, a method includes transmitting, from a first transducer of an electronic device, a first audio signal to a surface near the electronic device. The first audio signal is generated based on a frequency sweep across a range of frequencies. The method also includes receiving, at a second transducer of the electronic device, a second audio signal that is at least partly reflected off the surface. The method then determines an attribute of the surface based on the received second audio signal.

Abstract: In one embodiment, an apparatus includes an electrode that is coupled to a body of a user and is configured to receive a signal from the body. The received signal is based on an electromagnetic interference signal generated by an object that is external to the apparatus. The apparatus further includes one or more processors coupled to the electrode. The processors are configured to detect, based on the signal received by the electrode, one or more of: an interaction between the user and the object, an identity of the object, or a context surrounding the apparatus.

Abstract: In one embodiment, a method includes receiving, by an electrode of a device, a signal from a user's body. The received signal is based on an electromagnetic interference signal generated by an object that is external to the device. The method further includes determining, using machine learning applied to the received signal, one or more of the following: an identity of the object, an interaction between the user and the object, or a context surrounding the device.