Abstract: A machine learning based image processing architecture and associated applications are disclosed herein. In some embodiments, a machine learning framework is trained to learn low level image attributes such as object/scene types, geometries, placements, materials and textures, camera characteristics, lighting characteristics, contrast, noise statistics, etc. Thereafter, the machine learning framework may be employed to detect such attributes in other images and process the images at the attribute level.

Abstract: Techniques for generating aligned, augmented reality views are disclosed. In some embodiments, an estimate of a floorplan of a three-dimensional space is first generated based on one or more views of the three-dimensional space from a camera of a device located in the three-dimensional space. A cost function is next minimized with respect to the estimate of the floorplan to generate a corrected floorplan having less error than the estimated floorplan, and a camera pose corresponding to the corrected floorplan is determined. Display of a rendering having the determined camera pose is facilitated in an actual view of the camera, resulting in an aligned, augmented reality view.

Abstract: Improved techniques for generating video content are disclosed. In some embodiments, a frame is rendered using a rendering option. It is then determined whether the frame is an independent frame (I-frame) or a predictive frame (P-frame). In the event that the frame is an I-frame, the frame is independently encoded. In the event that the frame is a P-frame, motion is estimated from a known geometry of an associated scene and motion vectors and prediction error are encoded.

Abstract: Techniques for generating an arbitrary view of a scene are disclosed. In some embodiments, for each image of a set of images associated with a scene, image pixels are divided into a plurality of pixel blocks, the pixel blocks are transformed into an arbitrary perspective, and pixels of an image array comprising the arbitrary perspective are populated with corresponding pixels from the transformed pixel blocks. An output image of the scene comprising the arbitrary perspective is then generated based at least in part on the populated image array.

Abstract: A machine learning based image processing architecture and associated applications are disclosed herein. In some embodiments, a machine learning framework is trained to learn low level image attributes such as object/scene types, geometries, placements, materials and textures, camera characteristics, lighting characteristics, contrast, noise statistics, etc. Thereafter, the machine learning framework may be employed to detect such attributes in other images and process the images at the attribute level.

Abstract: Techniques for generating three-dimensional content from the recordings of multiple independently operated cameras that are not constrained to fixed positions and orientations are disclosed. In some embodiments, data from a plurality of cameras configured to record a scene is received; a relative pose of each camera with respect to a common set of identified points in the scene is determined; relative poses of the cameras with respect to each other based on their relative poses with respect to the common set of identified points are determined; and the camera recordings and determined relative poses are processed to generate a three-dimensional reconstruction of at least a portion of the scene in the cameras' common field of view.

Abstract: A machine learning based image processing architecture and associated applications are disclosed herein. In some embodiments, a machine learning framework is trained to learn low level image attributes such as object/scene types, geometries, placements, materials and textures, camera characteristics, lighting characteristics, contrast, noise statistics, etc. Thereafter, the machine learning framework may be employed to detect such attributes in other images and process the images at the attribute level.

Abstract: Techniques for generating an arbitrary ensemble view from existing views of a plurality of scenes are disclosed. In some embodiments, for each of a plurality of scenes, each of a corresponding set of one or more views is normalized to a desired perspective if its existing perspective is different than the desired perspective. An ensemble view comprising the plurality of scenes that has the desired perspective is generated at least in part using pixels from views of the scenes having the desired perspective including the normalized views.

Abstract: Techniques for tagging virtualized content are disclosed. In some embodiments, a modeled three-dimensional scene of objects representing abstracted source content is generated and analyzed to determine a contextual characteristic of the scene that is based on a plurality of objects comprising the scene. The modeled scene is tagged with a tag specifying the determined contextual characteristic.

Abstract: Two-dimensional compositing that preserves the curvatures of non-flat surfaces is disclosed. In some embodiments, a mapping is associated with a two-dimensional rendering that maps a potentially variable portion of the two-dimensional rendering to a canvas. The mapping is generated from a three-dimensional model of the potentially variable portion of the two-dimensional rendering. The potentially variable portion of the two-dimensional rendering is dynamically modified according to the mapping to reflect content comprising the canvas or edits received with respect to the canvas.

Abstract: Various techniques associated with automatic mesh generation are disclosed. One or more center curves of an outline of an object or figure are first determined. Next, for each of a plurality of points of each of the one or more center curves, a pair of rays is cast from a center curve in opposite directions, wherein the rays collide with opposite sides of the outline, and a collision pair is generated that comprises a line connecting collision points of the pair of rays on opposite sides of the outline. A mesh model of the object or figure is generated by mapping each of a set of collision pairs to polygons used to define the mesh model.

Abstract: Techniques associated with adaptive sampling are disclosed. In some embodiments, in response to receiving a specification of a scene to render, a sampling of each portion of the scene is determined based at least in part on the specification of the scene such that the scene is not uniformly sampled, and the scene is rendered according to the determined sampling of each portion of the scene.

Abstract: Adaptive sampling of pixels is disclosed. In some embodiments, a rendering of a scene is generated by sampling each pixel of the rendering with a prescribed number of samples. Subsequently, those pixels of the rendering that do not satisfy a noise threshold are iteratively sampled with an additional sample during each iteration until all pixels of the rendering satisfy the noise threshold. The noise threshold is associated with noise arising due to under sampling. Pixels comprising the completed rendering are not uniformly sampled.

Abstract: Techniques for rendering a modeled scene are disclosed. In some embodiments, a database comprising locally available generic object definitions is maintained at a destination device at which a scene is desired to be rendered. A scene is rendered by configuring one or more locally available generic object definitions obtained from the database according to a received specification of a modeled scene, wherein rendering is constrained to rendering locally available objects whose definitions are included in the database.

Abstract: Improved techniques for generating video content are disclosed. In some embodiments, a frame is first rendered using a first rendering option. It is then determined whether the frame is an independent frame (I-frame) or a predictive frame (P-frame). In the event that the frame is an I-frame, the frame is rendered using a second rendering option. In the event that the frame is a P-frame, the frame as rendered by the first rendering option and a previous frame as rendered by the first rendering option are used to determine prediction error. The first rendering option comprises a lower computational complexity than the second rendering option.

Abstract: Techniques for tagging virtualized content are disclosed. In some embodiments, a modeled three-dimensional scene of objects representing abstracted source content is generated and analyzed to determine a contextual characteristic of the scene that is based on a plurality of objects comprising the scene. The modeled scene is tagged with a tag specifying the determined contextual characteristic.

Abstract: Techniques for virtualizing content are disclosed. One or more objects comprising source video content are determined. The one or more objects comprising the source video content are virtualized by mapping each to and representing each with a corresponding database object. Data comprising the corresponding database objects is provided for rendering the source video content instead of any original pixel information of the source video content so that a virtualized version of the source video content is rendered.

Abstract: Techniques for generating an arbitrary view from a plurality of other existing views are disclosed. In some embodiments, arbitrary view generation includes transforming each of a set of images of a scene into a requested perspective, wherein the set of images comprises perspectives that are different than the requested perspective, and generating an image of the scene comprising the requested perspective based at least in part on pixels harvested from the transformed set of images.

Abstract: Techniques for generating an arbitrary view from a plurality of other existing views are disclosed. In some embodiments, arbitrary view generation includes storing a set of images comprising a plurality of perspectives of an asset in a database and generating an image comprising a desired arbitrary perspective of the asset that is different than any of the plurality of perspectives by combining pixels comprising at least a subset of the set of images.

Abstract: Techniques for generating an arbitrary ensemble view from a plurality of other existing views are disclosed. In some embodiments, arbitrary view generation comprises retrieving a set of views corresponding to each of a plurality of objects, transforming the set of views of each of the plurality of objects to a requested perspective, and generating an ensemble view comprising the plurality of objects that has the requested perspective by combining pixels from the transformed sets of views of the plurality of objects.