Abstract: In particular embodiments, a computing system may receive a video comprising a plurality of image frames. The system may generate, for each image frame in the video, an initial depth map using a machine-learning model. The system may compute a misalignment error indicating depth misalignments in initial depth maps using a reprojection technique. The system may generate, for each image frame in the video, an optimized camera pose and a flexible deformation spline associated with the image frame to minimize the misalignment error. The system may generate, for each image frame in the video, a refined depth map by adjusting the initial depth map associated with the frame using the flexible deformation spline associated with the image frame.

Abstract: In one embodiment, a method includes accessing multiple 3D photos to be concurrently displayed through multiple frames positioned in a virtual space, each of the of 3D photos having an optimal viewing point in the virtual space and determining a reference point based on a head pose of a viewer within the virtual space. The method may further include adjusting each 3D photo by rotating the 3D photo so that the optimal viewing point of the 3D photo points at the reference point, translating the rotated 3D photo toward the reference point, and non-uniformly scaling the rotated and translated 3D photo based on a scaling factor determined using the reference point and a position of the frame through which the 3D photo is to be viewed. The method may further include rendering an image comprising the adjusted multiple 3D photos as seen through the multiple frames.

Abstract: In one embodiment, a method includes accessing multiple 3D photos to be concurrently displayed through multiple frames positioned in a virtual space, each of the of 3D photos having an optimal viewing point in the virtual space and determining a reference point based on a head pose of a viewer within the virtual space. The method may further include adjusting each 3D photo by rotating the 3D photo so that the optimal viewing point of the 3D photo points at the reference point, translating the rotated 3D photo toward the reference point, and non-uniformly scaling the rotated and translated 3D photo based on a scaling factor determined using the reference point and a position of the frame through which the 3D photo is to be viewed. The method may further include rendering an image comprising the adjusted multiple 3D photos as seen through the multiple frames.