Abstract: The method and system for labeling 2D data using a 3D model provides processes that include generating Neural Radiance Fields (NeRF) of a desired scene, converting the NeRF into a mesh, rendering images of the mesh in a simulator from a user-chosen new camera pose including position and orientation, labeling the newly rendered images where bounding boxes for the images are created, rendering images using the NeRF from the same user-chosen new camera pose, and overlaying the bounding boxes from the simulator onto the imagery from the NeRF. The method and system provide capability of augmenting a set of real images to create a more diverse training dataset that captures the scene and its objects from various new camera positions, while maintaining high qualities of images.

Abstract: The method and system for labeling 2D data using a 3D model provides processes that include generating Neural Radiance Fields (NeRF) of a desired scene, converting the NeRF into a mesh, rendering images of the mesh in a simulator from a user-chosen new camera pose including position and orientation, labeling the newly rendered images where bounding boxes for the images are created, rendering images using the NeRF from the same user-chosen new camera pose, and overlaying the bounding boxes from the simulator onto the imagery from the NeRF. The method and system provide capability of augmenting a set of real images to create a more diverse training dataset that captures the scene and its objects from various new camera positions, while maintaining high qualities of images.