Abstract: System, methods, and other embodiments described herein relate to a manner of training a depth prediction system using bounding boxes. In one embodiment, a method includes segmenting an image to mask areas beyond bounding boxes and identify unmasked areas within the bounding boxes. The method also includes training a depth model using depth losses from comparing weighted points associated with pixels of the image within the unmasked areas to ground-truth depth. The method also includes providing the depth model for object detection.

Abstract: Extended reality content in a video can be varied based on a risk level of an external environment of a vehicle. The video can be of an external environment of a vehicle can be presented in real-time on a display located within the vehicle. The display can be a video pass through display. The display can be an in-vehicle display, or it can be a part of a video pass-through extended reality headset. The video can present a view of an external environment of the vehicle as well as extended reality content. A risk level of the external environment of the vehicle can be determined. An amount of the extended reality content presented in the video can be varied based on the risk level.

Abstract: A method of forecasting risk-biased trajectories of agents surrounding an ego vehicle is described. The method includes sampling a risk-neutral latent space generated by a trained encoder of a generative network based on past surrounding agent trajectories. The method also includes predicting, based on the sampling of the risk-neutral latent space, risk-neutral future surrounding agent trajectories using a trained decoder of the generative network. The method further includes sampling a risk-biased latent space distribution generated by a trained, risk-aware encoder of the generative network based on past trajectories of the ego vehicle and a risk-sensitivity. The method also includes predicting, based on the sampling of the risk-biased latent space distribution, risk-biased future surrounding agent trajectories using the trained decoder of the generative network.

Abstract: A flexible tactile sensor includes a conductive target positioned in a first plane, at least three coils forming an array within a second plane, the second plane spaced apart from the first plane, a pliable material coupling the conductive target to the at least three coils, and an electronic device electrically coupled to each of the at least three coils, the electronic device configured to induce an AC signal within each of the at least three coils and measure a change in inductance in the at least three coils in response to movement of the conductive target.

Abstract: A system includes a camera configured to capture image data of an environment, a monitoring system configured to generate a gaze sequences of a subject, and a computing device communicatively coupled to the camera and the monitoring system. The computing device is configured to receive the image data from the camera and the gaze sequences from the monitoring system, implement a machine learning model comprising a convolutional encoder-decoder neural network configured to process the image data and a side-channel configured to inject the gaze sequences into a decoder stage of the convolutional encoder-decoder neural network, generate, with the machine learning model, a gaze probability density heat map, and generate, with the machine learning model, an attended awareness heat map.

Abstract: A cluster-supporting catalyst including porous carrier particles having acid sites, and catalyst metal clusters supported within the pores of the porous carrier particles. In the cluster-supporting catalyst including porous carrier particles having acid sites, and catalyst metal clusters supported within the pores of the porous carrier particles, the catalyst metal may be rhodium, the catalyst metal may be palladium, the catalyst metal may be platinum, or the catalyst metal may be copper.

Abstract: A method for localization performed by an agent includes receiving a query image of a current environment of the agent captured by a sensor integrated with the agent. The method also includes receiving a target image comprising a first set of keypoints matching a second set of keypoints of the query image. The first set of keypoints may be generated based on a task specified for the agent. The method still further includes determining a current location based on the target image.

Abstract: Structures and sensor assemblies having engagement structures for securing a compliant substrate assembly are disclosed. In one embodiment, a sensor assembly includes a compliant substrate assembly having a base layer, and a deformable layer heat-sealed to the base layer such that the base layer and the deformable layer define at least one inflatable chamber. The sensor assembly further includes a first member proximate to a first edge of the compliant substrate assembly, a second member proximate to a second edge of the compliant substrate assembly, wherein the second edge is opposite the first edge, and at least one pressure sensor fluidly coupled to the at least one inflatable chamber and operable to produce a signal indicative of a pressure within the at least one inflatable chamber.

Abstract: A machine learning system for predicting a time dependent property of a material system includes a processor and a memory communicably coupled to the processor. Stored in the memory is an acquisition module and a machine learning module. The machine learning module includes instructions that, when executed by the processor, cause the processor during each of one or more iterations, to train a machine learning model to learn an initial state vector, Hermitian operators encoding observables, and a Hamiltonian of a material system from the Schrödinger equation of the material system propagated in a time series. The machine learning model also predicts, based at least in part on the learned initial state vector, Hermitian operators, and Hamiltonian, at least one time dependent property of the material system at time not equal to zero.

Abstract: The present disclosure provides neural fields for sparse novel view synthesis of outdoor scenes. Given just a single or a few input images from a novel scene, the disclosed technology can render new 360° views of complex unbounded outdoor scenes. This can be achieved by constructing an image-conditional triplanar representation to model the 3D surrounding from various perspectives. The disclosed technology can generalize across novel scenes and viewpoints for complex 360° outdoor scenes.

Abstract: A computing device configured to display a virtual representation of an environment of a robot includes a display device, a memory, and a processor coupled to the memory. The processor is configured to receive data from the one or more sensors of the robot with respect to an object within an environment of the robot. The processor is also configured to display a virtual representation of the object within a virtual mapping of the environment based on the data received from the one or more sensors. The processor is further configured to receive input data selecting the virtual representation of the object. The processor is also further configured to send instructions to the robot to act in response to the received input data.

Abstract: Systems and methods described herein support enhanced computer vision capabilities which may be applicable to, for example, autonomous vehicle operation. An example method includes generating a latent space and a decoder based on image data that includes multiple images, where each image has a different viewing frame of a scene. The method also includes generating a volumetric embedding that is representative of a novel viewing frame of the scene. The method includes decoding, with the decoder, the latent space using cross-attention with the volumetric embedding, and generating a novel viewing frame of the scene based on an output of the decoder.

Abstract: Systems and methods for modeling and predicting scene occupancy in an environment of a robot are disclosed herein. One embodiment processes past agent-trajectory data, map data, and sensor data using one or more encoder neural networks to produce combined encoded input data; generates a weights vector for a Gaussian Mixture Model (GMM) based on the combined encoded input data; produces a volumetric spatio-temporal representation of occupancy in an environment of a robot by generating, for a plurality of modes of the GMM in accordance with the weights vector, corresponding sample probability distributions of scene occupancy based on respective means and variances of the plurality of modes, wherein the respective means and variances sample coefficients of a set of learned basis functions; and controls the operation of the robot based, at least in part, on the volumetric spatio-temporal representation of occupancy.

Abstract: Systems and methods described herein support enhanced computer vision capabilities which may be applicable to, for example, autonomous vehicle operation. An example method includes An example method includes training a shared latent space and a first decoder based on first image data that includes multiple images, and training the shared latent space and a second decoder based on second image data that includes multiple images. The method also includes generating a volumetric embedding that is representative of a novel viewing frame the first scene. Further, the method includes decoding, with the first decoders, the shared latent space with the volumetric embedding, and generating the novel viewing frame of the first scene based on the output of the first decoder.

Abstract: Systems and methods described herein support enhanced computer vision capabilities which may be applicable to, for example, autonomous vehicle operation. An example method includes generating, through training, a shared latent space based on (i) image data that include multiple images, where each image has a different viewing frame of a scene, and (ii) first and second types of embeddings, and training a decoder based on the first type of embeddings. The method also includes generating an embedding based on the first type of embeddings that is representative of a novel viewing frame of the scene, decoding, with the decoder, the shared latent space using cross-attention with the generated embedding, and generating the novel viewing frame of the scene based on an output of the decoder.

Abstract: A method for representing atomic structures as Gaussian processes is described. The method includes mapping a crystal structure of chemical elements in a real space, in which atoms of the chemical elements are represented in a unit cell. The method also includes learning, by a machine learning model, a 3D embedding of each of the chemical elements in the real space according to the mapping of the crystal structure of the chemical elements. The method further includes training the machine learning model according to a representation of the atoms of the chemical elements in the unit cell based on the mapping of the crystal structure of the chemical elements. The method also includes predicting a material property corresponding to a point within the real space.

Abstract: An example method includes generating embeddings of image data that includes multiple images, where each image has a different viewpoints of a scene, generating a latent space and a decoder, wherein the decoder receives embeddings as input to generate an output viewpoint, for each viewpoint in the image data, determining a volumetric rendering view synthesis loss and a multi-view photometric loss, and applying an optimization algorithm to the latent space and the decoder over a number of epochs until the volumetric rendering view synthesis loss is within a volumetric threshold and the multi-view photometric loss is within a multi-view threshold.

Abstract: A method for scene synthesis from human motion is described. The method includes computing three-dimensional (3D) human pose trajectories of human motion in a scene. The method also includes generating contact labels of unseen objects in the scene based on the computing of the 3D human pose trajectories. The method further includes estimating contact points between human body vertices of the 3D human pose trajectories and the contact labels of the unseen objects that are in contact with the human body vertices. The method also includes predicting object placements of the unseen objects in the scene based on the estimated contact points.

Abstract: Systems and methods for performing three-dimensional multi-object tracking are disclosed herein. In one example, a method includes the steps of determining a residual based on augmented current frame detection bounding boxes, augmented previous frame detection bounding boxes, augmented current frame shape descriptors, and augmented previous frame shape descriptors and predicting an affinity matrix using the residual. The residual indicates a spatiotemporal and shape similarity between current detections in a current frame point cloud data and previous detections in a previous frame point cloud data. The affinity matrix indicates associations between the previous detections and the current detections, as well as the augmented anchors.

Abstract: A computer implemented method for determining optimal values for operational parameters for a model predictive controller for controlling a vehicle, can receive from a data store or a graphical user interface, ranges for one or more external parameters. The computer implemented method can determine optimum values for external parameters of the vehicle by simulating a vehicle operation across the ranges of the one or more operational parameters by solving a vehicle control problem and determining an output of the vehicle control problem based on a result for the simulated vehicle operation. A vehicle can include a processing component configured to adjust a control input for an actuator of the vehicle according to a control algorithm and based on the optimum values of the vehicle parameter as determined by the computer implemented method.