Abstract: Each image in a sequence of images includes three-dimensional locations of object features depicted in the image, and a first camera position of the camera when the image is captured. A gap is detected between first camera positions associated with a first continuous and first camera positions associated with a second continuous subset, the first camera positions associated with the second continuous subset adjusted to close the gap. A view path for a virtual camera is determined based on the first camera positions and the adjusted first camera positions. Second camera positions are determined for the virtual camera, for each of the second camera positions: one of the first camera positions associated with the sequence of images is selected and warped using the first camera position, the second camera position, and the three-dimensional locations of object features depicted in the selected image. A sequence of the warped images is outputted.

Abstract: An artificial reality system is configured to more accurately and efficiently construct a 3D virtual representation of a real-world environment from a set of 2D images. The system identifies points and/or lines within the images that define a plane along an orientation and then performs a planar sweep along a perpendicular path to identify surfaces in which the plane intersects with multiple points. Points that appear to be in the same plane are “collapsed” into a cohesive plane to conserve processing power by estimating and/or storing parameters for the cohesive plane, rather than all of the individual 3D points. In this way, the system also “averages out” random variation in the planar surface that would otherwise result from random noise in the estimation of the individual 3D points. The system may then generate a 3D map from a constrained alignment of all of the identified planes.

Abstract: In one embodiment, a method includes generating an outputted sequence of warped images from a captured sequence of images. Using this captured sequence of images, a computing system may determine one or more three-dimensional locations of object features and a corresponding camera position for each image in the captured sequence of images. Utilizing the camera positions for each image, the computing system may determine a view path representing the perspective of a virtual camera. The computing system may identify one or more virtual camera positions for the virtual camera located on the view path, and subsequently warp one or more images from the sequence of captured images to represent the perspective of the virtual camera at each of the respective virtual camera positions. This results in a sequence of warped images that may be outputted for viewing and interaction on a client device.

Abstract: In one embodiment, a method includes generating an outputted sequence of warped images from a captured sequence of images. Using this captured sequence of images, a computing system may determine one or more three-dimensional locations of object features and a corresponding camera position for each image in the captured sequence of images. Utilizing the camera positions for each image, the computing system may determine a view path representing the perspective of a virtual camera. The computing system may identify one or more virtual camera positions for the virtual camera located on the view path, and subsequently warp one or more images from the sequence of captured images to represent the perspective of the virtual camera at each of the respective virtual camera positions. This results in a sequence of warped images that may be outputted for viewing and interaction on a client device.

Abstract: An artificial reality system is configured to more accurately and efficiently construct a 3D virtual representation of a real-world environment from a set of 2D images. The system identifies points and/or lines within the images that define a plane along an orientation and then performs a planar sweep along a perpendicular path to identify surfaces in which the plane intersects with multiple points. Points that appear to be in the same plane are “collapsed” into a cohesive plane to conserve processing power by estimating and/or storing parameters for the cohesive plane, rather than all of the individual 3D points. In this way, the system also “averages out” random variation in the planar surface that would otherwise result from random noise in the estimation of the individual 3D points. The system may then generate a 3D map from a constrained alignment of all of the identified planes.

Abstract: In one embodiment, a method includes accessing an image of a physical world scene, detecting a number of straight-line segments in the accessed image, identifying a first vanishing point and a second vanishing point in the image, where each vanishing point corresponds to a sub-set of the number of straight-line segments, and where the first vanishing point and the second vanishing point are orthogonal, identifying a planar region in the accessed image represented by one or more straight-line segments associated with the first vanishing point and one or more straight-line segments associated with the second vanishing point, transforming a virtual object associated with the planar region based on one or more properties associated with the planar region, and displaying the transformed virtual object over the image.

Abstract: In one embodiment, a method includes accessing an image of a physical world scene, detecting a number of straight-line segments in the accessed image, identifying a first vanishing point and a second vanishing point in the image, where each vanishing point corresponds to a sub-set of the number of straight-line segments, and where the first vanishing point and the second vanishing point are orthogonal, identifying a planar region in the accessed image represented by one or more straight-line segments associated with the first vanishing point and one or more straight-line segments associated with the second vanishing point, transforming a virtual object associated with the planar region based on one or more properties associated with the planar region, and displaying the transformed virtual object over the image.

Abstract: To enable better sharing and preservation of immersive experiences, a graphics system reconstructs a three-dimensional scene from a set of images of the scene taken from different vantage points. The system processes each image to extract depth information therefrom and then stitches the images (both color and depth information) into a multi-layered panorama that includes at least front and back surface layers. The front and back surface layers are then merged to remove redundancies and create connections between neighboring pixels that are likely to represent the same object, while removing connections between neighboring pixels that are not. The resulting layered panorama with depth information can be rendered using a virtual reality (VR) system, a mobile device, or other computing and display platforms using standard rendering techniques, to enable three-dimensional viewing of the scene.

Abstract: To enable better sharing and preservation of immersive experiences, a graphics system reconstructs a three-dimensional scene from a set of images of the scene taken from different vantage points. The system processes each image to extract depth information therefrom and then stitches the images (both color and depth information) into a multi-layered panorama that includes at least front and back surface layers. The front and back surface layers are then merged to remove redundancies and create connections between neighboring pixels that are likely to represent the same object, while removing connections between neighboring pixels that are not. The resulting layered panorama with depth information can be rendered using a virtual reality (VR) system, a mobile device, or other computing and display platforms using standard rendering techniques, to enable three-dimensional viewing of the scene.

Abstract: To enable better sharing and preservation of immersive experiences, a graphics system reconstructs a three-dimensional scene from a set of images of the scene taken from different vantage points. The system processes each image to extract depth information therefrom and then stitches the images (both color and depth information) into a multi-layered panorama that includes at least front and back surface layers. The front and back surface layers are then merged to remove redundancies and create connections between neighboring pixels that are likely to represent the same object, while removing connections between neighboring pixels that are not. The resulting layered panorama with depth information can be rendered using a virtual reality (VR) system, a mobile device, or other computing and display platforms using standard rendering techniques, to enable three-dimensional viewing of the scene.

Abstract: Various technologies described herein pertain to generation of an output hyper-lapse video from an input video. A smoothed camera path can be computed based upon the input video. Further, output camera poses can be selected from the smoothed camera path for output frames of the output hyper-lapse video. One or more selected input frames from the input video can be chosen for an output frame. The selected input frames can be chosen based at least in part upon an output camera pose for the output frame. Moreover, the selected input frames can be combined to render the output frame. Choosing selected input frames from the input video and combining the selected input frames can be performed for each of the output frames of the output hyper-lapse video.

Abstract: An assembly includes a pair of image capture devices that capture 360-degree, stereo cubemap representation images of a scene. A controller generates a representation of the scene by correcting errors caused by placement of the image capture devices relative to each other in the assembly. The controller rotates an image from the image capture device to align objects in the image with objects in an image from the additional image capture device. Additionally, the controller replaces portions of an image from the image capture device including the additional image capture device with portions of an image from the additional image capture device. Additionally, the controller uses optical flow to cancel horizontal disparity and vertical disparity between images captured by the image capture device and by the additional image capture device.

Abstract: The claimed subject matter provides for systems and/or methods for identification of instances of an object of interest in 2D images by creating a database of 3D curve models of each desired instance and comparing an image of an object of interest against such 3D curve models of instances. The present application describes identifying and verifying the make and model of a car from a possibly single image—after the models have been populated with training data of test images of many makes and models of cars. In one embodiment, an identification system may be constructed by generating a 3D curve model by back-projecting edge points onto a visual hull reconstruction from silhouettes of an instance. The system and methods employ chamfer distance and orientation distance provides reasonable verification performance, as well as an appearance model for the taillights of the car to increase the robustness of the system.

Abstract: The claimed subject matter provides for systems and/or methods for identification of instances of an object of interest in 2D images by creating a database of 3D curve models of each desired instance and comparing an image of an object of interest against such 3D curve models of instances. The present application describes identifying and verifying the make and model of a car from a possibly single image—after the models have been populated with training data of test images of many makes and models of cars. In one embodiment, an identification system may be constructed by generating a 3D curve model by back-projecting edge points onto a visual hull reconstruction from silhouettes of an instance. The system and methods employ chamfer distance and orientation distance provides reasonable verification performance, as well as an appearance model for the taillights of the car to increase the robustness of the system.

Abstract: A mobile device having the capability of performing real-time location recognition with assistance from a server is provided. The approximate geophysical location of the mobile device is uploaded to the server. Based on the mobile device's approximate geophysical location, the server responds by sending the mobile device a message comprising a classifier and a set of feature descriptors. This can occur before an image is captured for visual querying. The classifier and feature descriptors are computed during an offline training stage using techniques to minimize computation at query time. The classifier and feature descriptors are used to perform visual recognition in real-time by performing the classification on the mobile device itself.

Abstract: Various technologies described herein pertain to generation of an output hyper-lapse video from an input video. A smoothed camera path can be computed based upon the input video. Further, output camera poses can be selected from the smoothed camera path for output frames of the output hyper-lapse video. One or more selected input frames from the input video can be chosen for an output frame. The selected input frames can be chosen based at least in part upon an output camera pose for the output frame. Moreover, the selected input frames can be combined to render the output frame. Choosing selected input frames from the input video and combining the selected input frames can be performed for each of the output frames of the output hyper-lapse video.

Abstract: A document that includes a representation of a two-dimensional (2-D) image may be obtained. A selection indicator indicating a selection of at least a portion of the 2-D image may be obtained. A match correspondence may be determined between the selected portion of the 2-D image and a three-dimensional (3-D) image object stored in an object database, the match correspondence based on a web crawler analysis result. A 3-D rendering of the 3-D image object that corresponds to the selected portion of the 2-D image may be initiated.

Abstract: Described are techniques for image deconvolution to deblur an image given a blur kernel. Localized color statistics derived from the image to be deblurred serve as a prior constraint during deconvolution. A pixel's color is formulated as a linear combination of the two most prevalent colors within a neighborhood of the pixel. This may be repeated for many or all pixels in an image. The linear combinations of the pixels serve as a two-color prior for deconvolving the blurred image. The two-color prior is responsive to the content of the image and it may decouple edge sharpness from edge strength.

Abstract: A process for compressing and decompressing non-keyframes in sequential sets of contemporaneous video frames making up multiple video streams where the video frames in a set depict substantially the same scene from different viewpoints. Each set of contemporaneous video frames has a plurality frames designated as keyframes with the remaining being non-keyframes. In one embodiment, the non-keyframes are compressed using a multi-directional spatial prediction technique. In another embodiment, the non-keyframes of each set of contemporaneous video frames are compressed using a combined chaining and spatial prediction compression technique. The spatial prediction compression technique employed can be a single direction technique where just one reference frame, and so one chain, is used to predict each non-keyframe, or it can be a multi-directional technique where two or more reference frames, and so chains, are used to predict each non-keyframe.

Abstract: Over the past few years there has been a dramatic proliferation of digital cameras, and it has become increasingly easy to share large numbers of photographs with many other people. These trends have contributed to the availability of large databases of photographs. Effectively organizing, browsing, and visualizing such .seas. of images, as well as finding a particular image, can be difficult tasks. In this paper, we demonstrate that knowledge of where images were taken and where they were pointed makes it possible to visualize large sets of photographs in powerful, intuitive new ways. We present and evaluate a set of novel tools that use location and orientation information, derived semi-automatically using structure from motion, to enhance the experience of exploring such large collections of images.