Abstract: A method for training a machine learning (ML) model for generating a joint particle-based representation of a scene from multi-modality sensor data. The method includes receiving sensor data acquired by a set of optical sensors of different sensor modality, representing the same scene. A joint particle-based representation of the scene covered by the optical sensors is generated. Intersections between rays emanating from a sensor location with the joint particle-based representation are determined, based on which an observation of the scene is rendered. A sensor-specific loss function value is determined by comparing the rendered observation with an observation rendered based on the received sensor data. The ML model is trained by determining the intersections, rendering the observation and determining the value of the sensor-specific loss function for every optical sensor, and by optimizing a combination of the sensor-specific loss functions.

Abstract: A method for training a neural network that predicts one or more characteristic values of a specified scenery.