Abstract: 2D and 3D data of a scene are linked by associating points in the 3D data with corresponding points in multiple different 2D images within the 2D data. Labels assigned to points in either data can be propagated to the other data. Labels propagated to a point in the 3D data are aggregated, and the labels ranked highest are kept and propagated back to the 2D images. 3D data including labels produced in this manner allow partially obscured objects in certain views to be more accurately identified. Thus, an object can be manipulated in all 2D views of the 2D data in which the object is at least partially visible, in order to digitally remove, alter, or replace it.