Abstract: Methods, computer systems, and apparatus, including computer programs encoded on computer storage media, for generating label data for one or more target objects in an environment. The system obtains first data characterizing the environment, wherein the first data includes position data characterizing a position of the target object. The system obtains second data including one or more three-dimensional (3D) frames characterizing the environment. The system determines, based on the first data, a guide feature for locating the target object in the 3D frames of the second data. The system receives a first user input that specifies at least an object position in the selected 3D frame, and generates label data for the target object based on the first user input.

Abstract: Various embodiments of the disclosure are directed to a scene reconstruction and machine learning system. In embodiments, the system comprises a storage medium configured to store image data, one or more scene models, one or more relightable matter fields, and information related to a machine learning model. In one or more embodiments, the system comprises an input circuit configured to receive image data characterizing light in a scene. In embodiments the system includes a processor. In embodiments the processor configured to reconstruct a scene model representing the scene using the image data. In embodiments the processor is configured to extract a relightable matter field from the scene model representing the object, store the scene model and the relightable matter field representing the object in the storage medium, apply the relightable matter field as an input to the machine learning model, and generate an output from the machine learning model.

Abstract: Examples disclosed herein involve a computing system configured to (i) receive sensor data associated with a vehicle's period of operation in an environment including (a) trajectory data associated with the vehicle and (b) at least one of trajectory data associated with one or more agents in the environment or data associated with one or more static objects in the environment, (ii) determine that at least one of (a) the one or more agents or (b) the one or more static objects is relevant to the vehicle, (iii) identify one or more times when there is a change to the one or more agents or the one or more static objects relevant to the vehicle, (iv) designate each identified time as a boundary point that separates the period of operation into one or more scenes, and (v) generate a representation of the one or more scenes based on the designated boundary points.