Abstract: A method includes obtaining surface samples that represent three-dimensional locations of surfaces of an environment; generating a voxelized representation of the surfaces of the environment in three-dimensional space using the surface samples; obtaining an image that shows the surfaces of the environment; associating each of the surface samples with image information that corresponds to a portion of the image that is spatially correlated with a respective one of the surface samples; determining voxel features for voxels from the voxelized representation based on the surface samples and the image information using a first trained machine learning model, wherein the voxel features each describe three-dimensional shapes present within a respective one of the voxels; and detecting objects based on the voxel features.