Abstract: In various examples, one or more systems detect, via received sensor data, intervention or disengagement events associated with one or more autonomous or semi-autonomous robotics systems included in a fleet of robotics systems. Based on a detected intervention or disengagement event, the one or more systems may generate an alert requesting human assistance or guidance. The one or more systems may also analyze a detected intervention or disengagement event and generate a natural language description of the event, classify the event into one or more categories and/or failure modes, generate novel entries in a training dataset based on the detected event, or initiate an automated retraining of one or more autonomous or semi-autonomous control routines based on the detected event.

Abstract: In various examples, a technique for performing end-to-end navigation using a generative world model includes converting a set of sensory inputs received by a machine at a current time step into a set of embedded features. The technique also includes generating, via execution of one or more neural networks, one or more states associated with the current time step based at least on the set of embedded features, a history of states preceding the current time step, and a first set of actions associated with a previous time step. The technique further includes converting, via execution of the one or more neural networks, the one or more states into a set of predictions associated with the current time step, and performing, by the machine, a second set of actions associated with the current time step based on the set of predictions.

Abstract: Robotic control systems that include a diffusion model configured by training on demonstration videos of tasks performed by humans, the diffusion model configured to transform a noise pattern and pose of an object manipulated in a task into a prediction of a next pose of the object in the task, and the system configured to generate an ending pose prediction for the task.

Abstract: In various examples, a technique for performing a task includes converting one or more sensory inputs obtained using one or more sensors of a machine into a plurality of segments. The technique also includes, for each segment included in the plurality of segments, generating, via execution of a machine learning model, a caption for the segment, and storing, in a data store, a representation of the caption in association with the segment. The technique further includes performing, by the machine, one or more actions based at least on one or more queries of the data store.

Abstract: An autonomous mobile device moves through a physical space using simultaneous localization and mapping (SLAM) techniques. SLAM processes images from cameras to determine localization and trajectory of the device based on features that are assumed to be stationary. SLAM performance is improved by removing moving features from consideration. A first position of a feature at a first time and data from an inertial sensor are used to determine a predicted position at a second time. The predicted position is compared to a second position of the feature at the second time. This comparison takes into consideration an assumed Gaussian error distribution of how the positions are determined. If the predicted position differs from the second position by less than a threshold value, the feature may be determined to be stationary. The stationary features are then processed using SLAM to determine the localization and trajectory information.

Abstract: Embodiments of the present invention relates to a method, apparatus and computer program product for processing, analyzing and/or display data related to performance metrics, for example, predicted performance metrics. In some embodiments, the apparatus includes one or more computers, a user input device(s), and a display device(s). In some embodiments, the one or more computers are operative, to compute, from historical KPI-related data, predicted future KPI-related behavior. In some embodiments, the display device is operative to display indications of predicted KPI values and/or predicted influencer values and/or a predicted future influencer-ranking-hierarchy and/or calculated deviating influencer values and/or one or more of the deviating future KPI values. In some embodiments, one or more user input devices are configured to receive indications of user-proposed influencer values and/or any other relevant data.