Abstract: A method of calibrating multiple cameras includes determining a first position of an overlay within a first field of view of a first camera and a second position of the overlay within a second field of view of a second camera; selecting, from the first camera and the second camera based on the first position and the second position, the first camera as a presentation camera; presenting, on a display, an image of a calibration object captured by the presentation camera and the overlay, the overlay indicating a calibration position for the calibration object with respect to the presentation camera; capturing a first calibration image of the calibration object by the first camera and a second calibration image of the calibration object by the second camera; and calibrating the first camera based on the first calibration image and calibrating the second camera based on the second calibration image.

Abstract: A telepresence system may include a display configured to present three-dimensional images. The 3D images may be rendered from multiple images captured by multiple cameras that image an area from different viewpoints. Misalignment of any of the multiple cameras may negatively affect the rendering. Accordingly, the telepresence system may calibrate the cameras to compensate for any misalignment as part of the rending. This calibration may include capturing an image, or images, of a calibration target to determine the relative positions of the cameras.

Abstract: A multi-camera display for a telepresence system may include a plurality of cameras aligned around a display panel large enough to display a person at scale. Aligning the cameras to particular directions can allow for images captured by the cameras to be combined to render a 3D image. The quality of the rendering may decrease when any or all of the cameras are misaligned. The present disclosure describes a system and method to test a frame assembly for a multi-camera display that uses mirrors and lasers to sense the camera alignments and targets to visually inspect the quality of the sensed camera alignments. Testing using this approach may simplify the testing because the alignments may be visually tested simultaneously, and the testing may be performed before the cameras are installed.

Abstract: A method of calibrating multiple cameras includes determining a first position of an overlay within a first field of view of a first camera and a second position of the overlay within a second field of view of a second camera; selecting, from the first camera and the second camera based on the first position and the second position, the first camera as a presentation camera; presenting, on a display, an image of a calibration object captured by the presentation camera and the overlay, the overlay indicating a calibration position for the calibration object with respect to the presentation camera; capturing a first calibration image of the calibration object by the first camera and a second calibration image of the calibration object by the second camera; and calibrating the first camera based on the first calibration image and calibrating the second camera based on the second calibration image.

Abstract: Techniques include a calibration assembly for a telepresence system that includes a stereoscopic display and a set of cameras. The calibration assembly may include at least one chart having chart markers, a mirror having mirror markers, and a processor. An example calibration assembly has three charts and the mirror attached to one of the charts. During calibration, the display is configured to display a set of display markers that are imaged in the mirror. Each camera forms a respective image of the set of chart markers, the set of mirror markers, and the set of display markers. The processing circuitry then determines the poses of the cameras with respect to the display based on the images of the set of chart markers, the set of mirror markers, and the set of display markers.

Abstract: A telepresence system may include a display configured to present three-dimensional images. The 3D images may be rendered from multiple images captured by multiple cameras that image an area from different viewpoints. Misalignment of any of the multiple cameras may negatively affect the rendering. Accordingly, the telepresence system may calibrate the cameras to compensate for any misalignment as part of the rending. This calibration may include capturing an image, or images, of a calibration target to determine the relative positions of the cameras.

Abstract: A neural network model receives color data for a sequence of images corresponding to a dynamic scene in three-dimensional (3D) space. Motion of objects in the image sequence results from a combination of a dynamic camera orientation and motion or a change in the shape of an object in the 3D space. The neural network model generates two components that are used to produce a 3D motion field representing the dynamic (non-rigid) part of the scene. The two components are information identifying dynamic and static portions of each image and the camera orientation. The dynamic portions of each image contain motion in the 3D space that is independent of the camera orientation. In other words, the motion in the 3D space (estimated 3D scene flow data) is separated from the motion of the camera.

Abstract: A neural network model receives color data for a sequence of images corresponding to a dynamic scene in three-dimensional (3D) space. Motion of objects in the image sequence results from a combination of a dynamic camera orientation and motion or a change in the shape of an object in the 3D space. The neural network model generates two components that are used to produce a 3D motion field representing the dynamic (non-rigid) part of the scene. The two components are information identifying dynamic and static portions of each image and the camera orientation. The dynamic portions of each image contain motion in the 3D space that is independent of the camera orientation. In other words, the motion in the 3D space (estimated 3D scene flow data) is separated from the motion of the camera.

Abstract: A neural network model receives color data for a sequence of images corresponding to a dynamic scene in three-dimensional (3D) space. Motion of objects in the image sequence results from a combination of a dynamic camera orientation and motion or a change in the shape of an object in the 3D space. The neural network model generates two components that are used to produce a 3D motion field representing the dynamic (non-rigid) part of the scene. The two components are information identifying dynamic and static portions of each image and the camera orientation. The dynamic portions of each image contain motion in the 3D space that is independent of the camera orientation. In other words, the motion in the 3D space (estimated 3D scene flow data) is separated from the motion of the camera.

Abstract: Missing image content is generated using a neural network. In an embodiment, a high resolution image and associated high resolution semantic label map are generated from a low resolution image and associated low resolution semantic label map. The input image/map pair (low resolution image and associated low resolution semantic label map) lacks detail and is therefore missing content. Rather than simply enhancing the input image/map pair, data missing in the input image/map pair is improvised or hallucinated by a neural network, creating plausible content while maintaining spatio-temporal consistency. Missing content is hallucinated to generate a detailed zoomed in portion of an image. Missing content is hallucinated to generate different variations of an image, such as different seasons or weather conditions for a driving video.

Abstract: Missing image content is generated using a neural network. In an embodiment, a high resolution image and associated high resolution semantic label map are generated from a low resolution image and associated low resolution semantic label map. The input image/map pair (low resolution image and associated low resolution semantic label map) lacks detail and is therefore missing content. Rather than simply enhancing the input image/map pair, data missing in the input image/map pair is improvised or hallucinated by a neural network, creating plausible content while maintaining spatio-temporal consistency. Missing content is hallucinated to generate a detailed zoomed in portion of an image. Missing content is hallucinated to generate different variations of an image, such as different seasons or weather conditions for a driving video.

Abstract: A method, computer readable medium, and system are disclosed for generating a Gaussian mixture model hierarchy. The method includes the steps of receiving point cloud data defining a plurality of points; defining a Gaussian Mixture Model (GMM) hierarchy that includes a number of mixels, each mixel encoding parameters for a probabilistic occupancy map; and adjusting the parameters for one or more probabilistic occupancy maps based on the point cloud data utilizing a number of iterations of an Expectation-Maximum (EM) algorithm.

Abstract: A neural network model receives color data for a sequence of images corresponding to a dynamic scene in three-dimensional (3D) space. Motion of objects in the image sequence results from a combination of a dynamic camera orientation and motion or a change in the shape of an object in the 3D space. The neural network model generates two components that are used to produce a 3D motion field representing the dynamic (non-rigid) part of the scene. The two components are information identifying dynamic and static portions of each image and the camera orientation. The dynamic portions of each image contain motion in the 3D space that is independent of the camera orientation. In other words, the motion in the 3D space (estimated 3D scene flow data) is separated from the motion of the camera.

Abstract: A method, computer readable medium, and system are disclosed for generating a Gaussian mixture model hierarchy. The method includes the steps of receiving point cloud data defining a plurality of points; defining a Gaussian Mixture Model (GMM) hierarchy that includes a number of mixels, each mixel encoding parameters for a probabilistic occupancy map; and adjusting the parameters for one or more probabilistic occupancy maps based on the point cloud data utilizing a number of iterations of an Expectation-Maximum (EM) algorithm.