Abstract: A system and method for generating realistic depth images by enhancing simulated images rendered from a 3D model, include a rendering engine configured to render noiseless 2.5D images by rendering various poses with respect to a target 3D CAD model, a noise transfer engine configured to apply realistic noise to the noiseless 2.5D images, and a background transfer engine configured to add pseudo-realistic scenedependent backgrounds to the noiseless 2.5D images. The noise transfer engine is configured to learn noise transfer based on a mapping, by a first generative adversarial network (GAN), of the noiseless 2.5D images to real 2.5D scans generated by a targeted sensor. The background transfer engine is configured to learn background generation based on a processing, by a second GAN, of output data of the first GAN as input data and corresponding real 2.5D scans as target data.

Abstract: A method of removing noise from a depth image includes presenting real-world depth images in real-time to a first generative adversarial neural network (GAN), the first GAN being trained by synthetic images generated from computer assisted design (CAD) information of at least one object to be recognized in the real-world depth image. The first GAN subtracts the background in the real-world depth image and segments the foreground in the real-world depth image to produce a cleaned real-world depth image. Using the cleaned image, an object of interest in the real-world depth image can be identified via the first GAN trained with synthetic images and the cleaned real-world depth image. In an embodiment the cleaned real-world depth image from the first GAN is provided to a second GAN that provides additional noise cancellation and recovery of features removed by the first GAN.

Abstract: A system and method for generating realistic depth images by enhancing simulated images rendered from a 3D model, include a rendering engine configured to render noiseless 2.5D images by rendering various poses with respect to a target 3D CAD model, a noise transfer engine configured to apply realistic noise to the noiseless 2.5D images, and a background transfer engine configured to add pseudo-realistic scenedependent backgrounds to the noiseless 2.5D images. The noise transfer engine is configured to learn noise transfer based on a mapping, by a first generative adversarial network (GAN), of the noiseless 2.5D images to real 2.5D scans generated by a targeted sensor. The background transfer engine is configured to learn background generation based on a processing, by a second GAN, of output data of the first GAN as input data and corresponding real 2.5D scans as target data.