Abstract: A method for training a machine learning (ML) model for generating a joint particle-based representation of a scene from multi-modality sensor data. The method includes receiving sensor data acquired by a set of optical sensors of different sensor modality, representing the same scene. A joint particle-based representation of the scene covered by the optical sensors is generated. Intersections between rays emanating from a sensor location with the joint particle-based representation are determined, based on which an observation of the scene is rendered. A sensor-specific loss function value is determined by comparing the rendered observation with an observation rendered based on the received sensor data. The ML model is trained by determining the intersections, rendering the observation and determining the value of the sensor-specific loss function for every optical sensor, and by optimizing a combination of the sensor-specific loss functions.

Abstract: A method for training a neural network that predicts one or more characteristic values of a specified scenery.

Abstract: A computer-implemented method for classification of at least one object in an environment of a vehicle. The method includes: collecting first data from a first sensor within a first data collecting frame; collecting second data from at least a second sensor within a second data collecting frame; determining a first object representation using the first data; determining a second object representation using the second data; updating the first and/or second object representation depending on an arrival of third data from the at least second sensor collected in a third data collecting frame after the first data collecting frame; fusing the first and second representation to determine an updated representation of the object based on the received data; applying the updated representation for training the data-driven model as input data for a data-driven model to obtain output data containing an information about a classification of the detected object.

Abstract: A method for generating training data for an object detector. The method includes receiving a plurality of optical images of a scene, each camera showing the scene from a respective viewing direction of a plurality of different viewing directions, receiving a plurality of sensor data elements, each sensor data element including sensor data other than optical image data of the scene from a respective sensing direction of a plurality of different sensing directions, training a first neural radiance field using the plurality of optical images to generate, for each 3D point of the scene, a respective value of a predetermined feature, training a second neural radiance field using the plurality of sensor data elements to generate, for each 3D point of the scene, a respective sensor data value and generating training data elements for the object detector using the first and the second neural radiance field.