Abstract: In various examples, systems and methods are disclosed relating to aligning images into frames of a first video using at least one first temporal attention layer of a neural network model. The first video has a first spatial resolution. A second video having a second spatial resolution is generated by up-sampling the first video using at least one second temporal attention layer of an up-sampler neural network model, wherein the second spatial resolution is higher than the first spatial resolution.

Abstract: In various examples, sensor configuration for autonomous or semi-autonomous systems and applications is described. Systems and methods are disclosed that may use image feature correspondences between camera images along with an assumption that image features are locally planar to determine parameters for calibrating an image sensor with a LiDAR sensor and/or another image sensor. In some examples, an optimization problem is constructed that attempts to minimize a geometric loss function, where the geometric loss function encodes the notion that corresponding image features are views of a same point on a locally planar surface (e.g., a surfel or mesh) that is constructed from LiDAR data generated using a LiDAR sensor. In some examples, performing such processes to determine the calibration parameters may remove structure estimation from the optimization problem.

Abstract: Approaches presented herein provide for the optimization of tasks performed for an operation such as the rendering of an image. A Frame Interceptor (FI) can generate a resource dependency graph (RDG) by intercepting API calls during the rendering process and determining dependencies. FI can analyze the RDG to identify potential optimizations, such as may correspond to reordering or parallel execution of certain tasks. FI can automatically test optimizations to determine whether sufficient improvement is obtained. This testing can be performed in real time by replacing the originally intercepted API calls with the newly ordered API calls generated by FI. FI can then issue a report that indicates information such as the changes made, the time taken to render the image, and potentially the fact that the images were determined to be identical.

Abstract: In various examples, a three-dimensional (3D) intersection structure may be predicted using a deep neural network (DNN) based on processing two-dimensional (2D) input data. To train the DNN to accurately predict 3D intersection structures from 2D inputs, the DNN may be trained using a first loss function that compares 3D outputs of the DNN—after conversion to 2D space—to 2D ground truth data and a second loss function that analyzes the 3D predictions of the DNN in view of one or more geometric constraints—e.g., geometric knowledge of intersections may be used to penalize predictions of the DNN that do not align with known intersection and/or road structure geometries. As such, live perception of an autonomous or semi-autonomous vehicle may be used by the DNN to detect 3D locations of intersection structures from 2D inputs.

Abstract: A game-agnostic event detector can be used to automatically identify game events. Game-specific configuration data can be used to specify types of pre-processing to be performed on media for a game session, as well as types of detectors to be used to detect events for the game. Event data for detected events can be written to an event log in a form that is both human- and process-readable. The event data can be used for various purposes, such as to generate highlight videos or provide player performance feedback. The event data may be determined based upon output from detectors such as optical character recognition (OCR) engines, and the regions may be upscaled and binarized before OCR processing.

Abstract: In various examples, estimated field of view or gaze information of a user may be projected external to a vehicle and compared to vehicle perception information corresponding to an environment outside of the vehicle. As a result, interior monitoring of a driver or occupant of the vehicle may be used to determine whether the driver or occupant has processed or seen certain object types, environmental conditions, or other information exterior to the vehicle. For a more holistic understanding of the state of the user, attentiveness and/or cognitive load of the user may be monitored to determine whether one or more actions should be taken. As a result, notifications, AEB system activations, and/or other actions may be determined based on a more complete state of the user as determined based on cognitive load, attentiveness, and/or a comparison between external perception of the vehicle and estimated perception of the user.

Abstract: In various examples, high-precision semantic image editing for machine learning systems and applications are described. For example, a generative adversarial network (GAN) may be used to jointly model images and their semantic segmentations based on a same underlying latent code. Image editing may be achieved by using segmentation mask modifications (e.g., provided by a user, or otherwise) to optimize the latent code to be consistent with the updated segmentation, thus effectively changing the original, e.g., RGB image. To improve efficiency of the system, and to not require optimizations for each edit on each image, editing vectors may be learned in latent space that realize the edits, and that can be directly applied on other images with or without additional optimizations. As a result, a GAN in combination with the optimization approaches described herein may simultaneously allow for high precision editing in real-time with straightforward compositionality of multiple edits.

Abstract: Disclosed are systems and methods relating to extracting 3D features, such as bounding boxes. The systems can apply, to one or more features of a source image that depicts a scene using a first set of camera parameters, based on a condition view image associated with the source image, an epipolar geometric warping to determine a second set of camera parameters. The systems can generate, using a neural network, a synthetic image representing the one or more features and corresponding to the second set of camera parameters.

Abstract: Apparatuses, systems, and techniques to generate blue noise masks for real-time image rendering and enhancement. In at least one embodiment, a noise mask is generated and applied to one or more images to generate one or more enhanced images for image processing (e.g., real-time image rendering). In at least one embodiment, the noise mask is able to handle the temporal domain (e.g., add time to the spatial domain) to improve image quality when rendering images over multiple frames.

Abstract: In various examples, audio alerts of emergency response vehicles may be detected and classified using audio captured by microphones of an autonomous or semi-autonomous machine in order to identify travel directions, locations, and/or types of emergency response vehicles in the environment. For example, a plurality of microphone arrays may be disposed on an autonomous or semi-autonomous machine and used to generate audio signals corresponding to sounds in the environment. These audio signals may be processed to determine a location and/or direction of travel of an emergency response vehicle (e.g., using triangulation). Additionally, to identify siren types—and thus emergency response vehicle types corresponding thereto—the audio signals may be used to generate representations of a frequency spectrum that may be processed using a deep neural network (DNN) that outputs probabilities of alert types being represented by the audio data.

Abstract: A cluster storage system takes snapshots that are consistent across all storage nodes. The storage system can nearly instantaneously promote a set of consistent snapshots to their respective base volumes to restore the base volumes to be the same as the snapshots. Given these two capabilities, users can restore the system to a recovery point of the user's choice, by turning off storage service I/O, promoting the snapshots constituting the recovery point, rebooting their servers, and resuming storage service I/O.

Abstract: Apparatuses, systems, and techniques to determine a matrix multiplication algorithm for a matrix multiplication operation. In at least one embodiment, a matrix multiplication operation is analyzed to determine an appropriate matrix multiplication algorithm to perform the matrix multiplication algorithm.

Abstract: In various examples, systems and methods are disclosed relating to error concealment by replacing a lost video frame region with a chronological predecessor. Network packets including data corresponding to an encoded bitstream of a frame of a video stream can be received. In response to determining that at least one packet of the video stream has been lost, a region of the video frame corresponding to the lost network packet can be replaced with the same region of a previous frame of the video stream.

Abstract: In various examples, systems and methods are disclosed relating to improving perceived video quality through temporal redistribution of network packet payloads that may carry error mitigation data. A subset of network packets for an encoded video stream is identified from a sequence of network packets as corresponding to a region of a video frame of the encoded video stream. A transmission order for the sequence of network packets is determined based at least on the subset of network packets and one or more error correction packets corresponding to the sequence of network packets. The sequence of network packets is transmitted to a receiver client device according to the transmission order.

Abstract: In various examples, systems and methods are disclosed relating to bandwidth preservation through selective application of error mitigation techniques for video frame regions. A subset of network packets for a video stream are identified as corresponding to an encoded region of a video frame of the video stream. At least one error correction packet is transmitted for the subset that encodes the region of the video frame. The network packets and the at least one error correction packet can be transmitted to a receiver device.

Abstract: A system can include a memory and a processing device, operatively coupled to the memory, to perform operations including receiving a header block of an ordered set of blocks. The header block includes a header block payload and a first digest. The operations further include authenticating, based on the header block payload, the header block, and receiving a first data block of the ordered set of blocks. The first data block includes a first data block payload and a second digest. The operations further include authenticating, based on the first digest, the first data block, and processing the first data block payload.

Abstract: A machine-learning control system is trained to perform a task using a simulation. The simulation is governed by parameters that, in various embodiments, are not precisely known. In an embodiment, the parameters are specified with an initial value and expected range. After training on the simulation, the machine-learning control system attempts to perform the task in the real world. In an embodiment, the results of the attempt are compared to the expected results of the simulation, and the parameters that govern the simulation are adjusted so that the simulated result matches the real-world attempt. In an embodiment, the machine-learning control system is retrained on the updated simulation. In an embodiment, as additional real-world attempts are made, the simulation parameters are refined and the control system is retrained until the simulation is accurate and the control system is able to successfully perform the task in the real world.

Abstract: Traditionally, a software application is developed, tested, and then published for use by end users. Any subsequent update made to the software application is generally in the form of a human programmed modification made to the code in the software application itself, and further only becomes usable once tested, published, and installed by end users having the previous version of the software application. This typical software application lifecycle causes delays in not only generating improvements to software applications, but also to those improvements being made accessible to end users. To help avoid these delays and improve performance of software applications, deep learning models may be made accessible to the software applications for use in providing inferenced data to the software applications, which the software applications may then use as desired. These deep learning models can furthermore be improved independently of the software applications using manual and/or automated processes.

Abstract: To ensure proper operation (e.g., speed and/or function) of standard cells fabricated within an integrated circuit a minimum potential difference between the high and low power supply rails needs to be maintained. IR drop refers to a reduction in the potential difference between the power supply rails and is caused when the switching activity of cells that share a power supply rail is greater than can be provided at a particular time. Before fabrication, placement of the cells is reorganized within bounding box regions. Power density across the power rails within each bounding box is normalized based on spatial and temporal power density characteristics of each cell. The reorganization is IR aware and has minimal impact on timing and IR drop is mitigated because distributing current consumption between the supply rails reduces current spikes and IR drops.

Abstract: Systems and methods for cooling a datacenter are disclosed. In at least one embodiment, multiple parallel refrigerant paths are associated with one or more flow controllers to cool multiple computing devices associated therewith, so that one or more flow controllers can distribute equal measures of a liquid phase of refrigerant, relative to a vapor phase of a refrigerant, to such parallel refrigerant paths based in part on a cooling requirement from at least one of such multiple computing devices.