Abstract: Various implementations disclosed herein include devices, systems, and methods that refine a first 3D surface representation (e.g., a 3D point cloud or a 3D mesh) using a second 3D surface representation that includes a 3D geometric primitive. In some implementations, a first 3D surface representation of a physical environment is obtained including points at 3D locations determined based on data generated by a first sensor. In some implementations, a second 3D surface representation corresponding to at least a portion of the physical environment is obtained that includes at least one 3D geometric primitive. In some implementations, a determination whether to adjust the 3D locations of at least one point of the points of the first 3D surface representation is made based on the 3D geometric primitive, and the 3D locations of the at least one point is adjusted to align with the geometric primitive based on the determination.