Abstract: A mobile robot can be caused to move according to a planned trajectory. The mobile robot can be a vehicle. Information about agents in an environment of the mobile robot can be received from sensors. At a first time, a spatiotemporal graph can be produced. The spatiotemporal graph can represent relationships among the agents in the environment. The mobile robot can be one of the agents in the environment. Information from the spatiotemporal graph can be input to neural networks to produce information for a mixture of affine time-varying systems. The mixture of affine time-varying systems can represent an evolution of agent states of the agents. Using the mixture of affine time-varying systems and information associated with the first time, a prediction of the agent states at a second time can be calculated. The mobile robot can be caused to move according to the planned trajectory determined from the prediction.

Abstract: Systems, methods, and other embodiments described herein relate to improving trajectory forecasting in a device. In one embodiment, a method includes, in response to receiving sensor data about a surrounding environment of the device, identifying an object from the sensor data that is present in the surrounding environment. The method includes determining category probabilities for the object, the category probabilities indicating semantic classes for classifying the object and probabilities that the object belongs to the semantic classes. The method includes forecasting trajectories for the object based, at least in part, on the category probabilities and the sensor data. The method includes controlling the device according to the trajectories.

Abstract: In various examples, policy planning using behavior models for autonomous and semi-autonomous systems and applications is described herein. Systems and methods are disclosed that determine a policy for navigating a vehicle, such as a semi-autonomous vehicle or an autonomous vehicle (or other machine), where the policy allows for multistage reasoning that leverages future reactive behaviors of one or more other objects. For instance, a first behavior model (e.g., a trajectory tree) may be generated that represents candidate trajectories for the vehicle and one or more second behavior models (e.g., one or more scenario trees) may be generated that respectively represent future behaviors of the other object(s). The first behavior model and the second behavior model(s) may then be processed, such as in a closed-loop simulation based on a realistic data-driven traffic model, to determine the policy for navigating the vehicle.

Abstract: A multimedia system allocates, during encoding of a multimedia stream, bits to portions of frames based on quality metrics and bit usages for different quantization parameters (QPs). An encoder of the multimedia system encodes a frame in a first pass with a first QP and in a second pass with a second QP. A comparator of the multimedia system measures and compares quality metrics, such as mean squared error, for each portion of the frame encoded with the first QP and the second QP. The comparator compares the difference between the quality metrics for each portion encoded with each QP to a threshold. If the difference in quality metrics for a portion exceeds the threshold, the comparator selects the portion for inclusion in a subset of portions to be encoded with the second QP.

Abstract: A display system modifies display cycles of one or more displays to perform a system operation while avoiding visual perturbations at the one or more displays. The display system modifies, synchronizes, or both, blanking periods of the one or more displays such that blanking periods equal or exceed a blackout duration and overlap for at least the blackout duration. Then the system performs the system operation during an overlapping portion of the one or more blanking periods, where the system operation reduces availability of display data at the one or more displays.

Abstract: An encoder calculates a first local encoding parameter for a first block of video content based on one or more local metrics. The encoder modifies the first local encoding parameter based on one or more second local encoding parameters for one or more second blocks of video content that are adjacent to the first block of video content. The encoder then encodes the first block using the modified first local encoding parameter. In some cases, the local encoding parameters are quantization parameters used to quantize values of pixels or compression parameters used to compress values of the pixels. The local metric can include one or more of a target bit rate, a texture complexity, a contrast, an indicator of motion in the first block, and an importance map.

Abstract: A plurality of agent locations can be determined at a plurality of time steps by inputting a plurality of images to a perception algorithm that inputs the plurality of images and outputs agent labels and the agent locations. A plurality of first uncertainties corresponding to the agent locations can be determined at the plurality of time steps by inputting the plurality of agent locations to a filter algorithm that inputs the agent locations and outputs the plurality of first uncertainties corresponding to the plurality of agent locations. A plurality of predicted agent trajectories and potential trajectories corresponding to the predicted agent trajectories can be determined by inputting the plurality of agent locations at the plurality of time steps and the first uncertainties corresponding to the agent locations at the plurality of time steps to a variational autoencoder.

Abstract: In various examples, cost probability distributions corresponding to predicted locations of an object in an environment and potential locations for a machine in the environment and may be evaluated using corresponding observed costs corresponding to the machine and the object. The cost probability distributions may be evaluated based on comparing the observed costs to threshold values, which may be determined based on sampling a predicted cost function. A threshold value may be selected to provide false-positive rate and/or false-negative rate guarantees for anomaly detection. Control operations may be performed based on results of the evaluation of the cost probability distributions. For example, based on the results, a motion planner may reuse a planned trajectory for a future planning cycle (e.g., thereby avoiding re-planning computations) or generate and/or select a new planned trajectory (e.g., based at least on one or more anomalies being detected).

Abstract: In various examples, a motion planner include an analytical function to predict motion plans for a machine based on predicted trajectories of actors in an environment, where the predictions are differentiable with respect to parameters of a neural network of a motion predictor used to predict the trajectories. The analytical function may be used to determine candidate trajectories for the machine based on a predicted trajectory, to compute cost values for the candidate trajectories, and to select a reference trajectory from the candidate trajectories. For differentiability, a term of the analytical function may correspond to the predicted trajectory. A motion controller may use the reference trajectory to predict a control sequence for the machine using an analytical function trained to generate predictions that are differentiable with respect to at least one parameter of the analytical function used to compute the cost values.

Abstract: Systems, apparatuses, and methods for bit budgeting in video encode pre-analysis based on context and features are disclosed. A pre-encoder receives a video frame and evaluates each block of the frame for the presence of several contextual indicators. The contextual indicators can include memory colors, text, depth of field, and other specific objects. For each contextual indicator detected, a coefficient is generated and added with other coefficients to generate a final importance value for the block. The coefficients can be adjusted so that only a defined fraction of the picture is deemed important. The final importance value of the block is used to determine the bit budget for the block. The block bit budgets are provided to the encoder and used to influence the quantization parameters used for encoding the blocks.

Abstract: Apparatuses, systems, and techniques to generate trajectory predictions. In at least one embodiment, trajectory predictions are generated based on, for example, one or more neural networks.

Abstract: In various examples, techniques for generating simulations for autonomous machines and applications are described herein. Systems and methods are disclosed that use various models to generate simulations. For instance, a first model(s) may process input data, such as input data representing maps indicating the locations of objects and state history of the objects within the environment, to determine navigation goals for the objects. Additionally, a second model(s) may then process the input data and data representing the navigation goals in order to determine possible trajectories (e.g., action samples) for the objects within the environment. Furthermore, a third model(s) may process the input data to predict trajectories of the objects within the environment. The systems and methods may then use at least the possible trajectories and the predicted trajectories to simulate the motion (e.g., one or more trajectories) of one or more of the objects.

Abstract: Apparatuses, systems, and techniques to perform trajectory predictions within one or more images. In at least one embodiment, a processor comprises one or more circuits to cause one or more neural networks to perform trajectory predictions of two or more objects detected within a plurality of frames without tracking the two or more objects based, at least in part, on processing a sequence of data of the one or more objects as a whole.

Abstract: A feedback processing module includes a memory configured to store feedback received from an encoder. The feedback includes parameters associated with encoded graphics content generated by a graphics engine. The feedback processing module also includes a processor configured to generate configuration information for the graphics engine based on the feedback. The graphics engine is configured to execute a workload based on the configuration information. In some cases, the feedback processing module is also configured to receive feedback from a decoder that is used to decode the graphics content that is encoded by the encoder and generate the configuration information based on the feedback received from the decoder.

Abstract: Systems, methods, and other embodiments described herein relate to improving controls in a device according to risk. In one embodiment, a method includes, in response to receiving sensor data about a surrounding environment of the device, identifying objects from the sensor data that are present in the surrounding environment. The method includes generating a control sequence for controlling the device according to a risk-sensitivity parameter to navigate toward a destination while considering risk associated with encountering the objects defined by the risk-sensitivity parameter. The method includes controlling the device according to the control sequence.

Abstract: Systems, apparatuses, and methods for performing asynchronous memory clock changes on multiple displays are disclosed. From time to time, a memory clock frequency change is desired for a memory subsystem storing frame buffer(s) used to drive pixels to multiple displays. For example, when the real-time memory bandwidth demand differs from the memory bandwidth available with the existing memory clock frequency, a control unit tracks the vertical blanking interval (VBI) timing of a first display. Also, the control unit causes a second display to enter into panel self-refresh (PSR) mode. Once the PSR mode of the second display overlaps with a VBI of the first display, a memory clock frequency change, including memory training, is initiated. After the memory clock frequency change, the displays are driven by the frame buffer(s) in the memory subsystem at an updated frequency.

Abstract: A display system modifies display cycles of one or more displays to perform a system operation while avoiding visual perturbations at the one or more displays. The display system modifies, synchronizes, or both, blanking periods of the one or more displays such that blanking periods equal or exceed a blackout duration and overlap for at least the blackout duration. Then the system performs the system operation during an overlapping portion of the one or more blanking periods, where the system operation reduces availability of display data at the one or more displays.

Abstract: A plurality of agent locations can be determined at a plurality of time steps by inputting a plurality of images to a perception algorithm that inputs the plurality of images and outputs agent labels and the agent locations. A plurality of first uncertainties corresponding to the agent locations can be determined at the plurality of time steps by inputting the plurality of agent locations to a filter algorithm that inputs the agent locations and outputs the plurality of first uncertainties corresponding to the plurality of agent locations. A plurality of predicted agent trajectories and potential trajectories corresponding to the predicted agent trajectories can be determined by inputting the plurality of agent locations at the plurality of time steps and the first uncertainties corresponding to the agent locations at the plurality of time steps to a variational autoencoder.

Abstract: A processing device for executing a machine learning neural network operation includes memory and a processor. The processor is configured to receive input data at a layer of the machine learning neural network operation, receive a plurality of sorted filters to be applied to the input data, apply the plurality of sorted filters to the input data to produce a plurality of different feature maps, compress the plurality of different feature maps according to a similarity of the feature maps relative to each other and store the plurality of different feature maps in the memory.

Abstract: The present disclosure is directed to techniques for determining a perceptual importance map. The perceptual importance map indicates the relative importance to the human visual system of different portions of an image. The techniques include obtaining cost values for the blocks of an image, where cost values are values used in determining motion vectors. For each block, a confidence value is derived from the cost values. The confidence value indicates the confidence with which the motion vector is believed to be correct. A perceptual importance value is determined based on the confidence value via one or more modifications to the confidence value to better reflect importance to the human visual system. The generated perceptual importance values can be used for various purposes such as allocating bits for encoding, identifying regions of interest, or selectively rendering portions of an image with greater or lesser detail based on relative perceptual importance.