Abstract: Embodiments can provide a strategy for controlling information flow both from known opacity regions to unknown regions, as well as within the unknown region itself. This strategy is formulated through the use and refinement of various affinity definitions. As a result of this strategy, a final linear system can be obtained, which can be solved in closed form. One embodiment pertains to identifying opacity information flows. The opacity information flow may include one or more of flows from pixels in the image that have similar colors to a target pixel, flows from pixels in the foreground and background to the target pixel, flows from pixels in the unknown opacity region in the image to the target pixel, flows from pixels immediately surrounding the target pixels in the image to the target pixel, and any other flow.

Abstract: One or more levels of maladaptation are calculated relative to frames of media content having abrupt jumps from periods of low illumination to bright illumination when visual acuity may be lost and/or discomfort may be experienced. These levels of maladaptation may be correlated with subjectively determined levels of perceived luminance discomfort. Based upon the levels of perceived luminance discomfort that can be derived from the levels of maladaptation, the media content may be adjusted.

Abstract: A simulation adapter can be attached to a real-world physical object and adapted to communicate one or more characteristics and/or simulation events associated with the real-world physical object to a simulation device. The simulation device is adapted to generate a simulation experience based upon the one or more characteristics of the real-world physical object and/or the simulation events associated with the real-world physical object.

Abstract: Systems for electronic media distribution includes a differential versioning server configured to receive a first media file including a first set of data with a first set of attributes and a second media file including a second set of data with a second set of attributes, generate a first differential data file as a function of differences between the first media file and the second media file, and generate a first differential metadata file including an encoding data set configured to enable a media decoder to regenerate the second media file by applying the first differential data file to the first media file. Systems for electronic media distribution may also include a receiver communicatively coupled to a differential versioning decoder configured to receive the first media file and the first differential data file and generate the second media file by applying the first differential data file to the first media file.

Abstract: Systems and methods are disclosed for weighting the image quality prediction of any visual-attention-agnostic quality metric with a saliency map. By accounting for the salient regions of an image or video frame, the disclosed systems and methods may dramatically improve the precision of the visual-attention-agnostic quality metric during image or video quality assessment. In one implementation, a method of saliency-weighted video quality assessment includes: determining a per-pixel image quality vector of an encoded video frame; determining per-pixel saliency values of the encoded video frame or a reference video frame corresponding to the encoded video frame; and computing a saliency-weighted image quality metric of the encoded video frame by weighting the per-pixel image quality vector using the per-pixel saliency values.

Abstract: An apparatus such as a head-mounted display (HMD) may have a camera for capturing a visual scene for presentation via the HMD. A user of the apparatus may be operating a second, physical camera for capturing video or still images within the visual scene. The HMD may generate an augmented reality (AR) experience by presenting an AR view frustum representative of the actual view frustum of the physical camera. The field of view of the user viewing the captured visual scene via the AR experience is generally larger that the AR view frustum, allowing the user to avoid unnatural tracking movements and/or hunting for subjects.

Abstract: The present disclosure relates to techniques for reconstructing an object in three dimensions that is captured in a set of two-dimensional images. The object is reconstructed in three dimensions by computing depth values for edges of the object in the set of two-dimensional images. The set of two-dimensional images may be samples of a light field surrounding the object. The depth values may be computed by exploiting local gradient information in the set of two-dimensional images. After computing the depth values for the edges, depth values between the edges may be determined by identifying types of the edges (e.g., a texture edge, a silhouette edge, or other type of edge). Then, the depth values from the set of two-dimensional images may be aggregated in a three-dimensional space using a voting scheme, allowing the reconstruction of the object in three dimensions.

Abstract: The present disclosure relates to techniques for reconstructing an object in three dimensions that is captured in a set of two-dimensional images. The object is reconstructed in three dimensions by computing depth values for edges of the object in the set of two-dimensional images. The set of two-dimensional images may be samples of a light field surrounding the object. The depth values may be computed by exploiting local gradient information in the set of two-dimensional images. After computing the depth values for the edges, depth values between the edges may be determined by identifying types of the edges (e.g., a texture edge, a silhouette edge, or other type of edge). Then, the depth values from the set of two-dimensional images may be aggregated in a three-dimensional space using a voting scheme, allowing the reconstruction of the object in three dimensions.

Abstract: Methods, systems, and computer-readable memory are provided for determining time-varying anatomical and physiological tissue characteristics of an animation rig. For example, shape and material properties are defined for a plurality of sample configurations of the animation rig. The shape and material properties are associated with the plurality of sample configurations. An animation of the animation rig is obtained, and one or more configurations of the animation rig are determined for one or more frames of the animation. The determined one or more configurations include shape and material properties, and are determined using one or more sample configurations of the animation rig. A simulation of the animation rig is performed using the determined one or more configurations. Performing the simulation includes computing physical effects for addition to the animation of the animation rig.

Abstract: The disclosure describes a high dynamic range video coding pipeline that may reduce color artifacts and improve compression efficiency. The disclosed pipeline separates the luminance component from the chrominance components of an input signal (e.g., an RGB source video) and applies a scaling of the chrominance components before encoding, thereby reducing perceivable color artifacts while maintaining luminance quality.

Abstract: Techniques for selectively removing Monte Carlo (MC) noise from a geometric buffer (G-buffer). Embodiments identify the G-buffer for rendering an image of a three-dimensional scene from a viewpoint. Embodiments determine, for each of a plurality of pixels in the image being rendered, respective world position information based on the three-dimensional scene and a position and orientation of the viewpoint. A pre-filtering operation is then performed to selectively remove the MC noise from the G-buffer, based on the determined world position information for the plurality of pixels.

Abstract: Systems and methods are described for recording high fidelity augmented reality or virtual reality experiences through off-device computation. In one implementation, an augmented reality system renders an augmented reality object overlaid over a real-world environment; captures video and audio data of the real-world environment during rendering of the augmented reality object; and stores augmented reality object data associated with the rendered augmented reality object. The augmented reality system transmits the augmented reality object data to a server that rerenders the augmented reality object based on the transmitted augmented reality object data to create rerendered augmented reality object data. The augmented reality system receives the rerendered augmented reality object data or a video of the rerendered augmented reality object data composited with the captured video and audio data.

Abstract: One particular implementation of the present invention may take the form of a method and apparatus for providing various movements as input to a video game. The method and apparatus may detect the body movements of a video game player and interpret those movements as inputs to the video game. The video game may then compare the movements of the user to expected movements to determine if the correct movement was performed by the user. The video game may also display a video game environment that instructs and interacts with the user to perform several movements of the video game. In this manner, the method and apparatus may provide a move realistic and dynamic video game experience than previous video games that only allowed the player to press buttons as inputs to the game.

Abstract: Systems and methods are provided for providing a platform to provide virtual storefronts to consumers. Environmental elements are associated with specific consumer services on computer server. A user in the environment takes audio or visual recordings of an environmental element and uploads the recordings to the server. The server determines the appropriate consumer service associated with the recorded environmental element and provides the user with a reference to the service. Accordingly, any space, such as a bus stop, subway or train terminal, movie theater, or airport, may be turned into a virtual store.

Abstract: A method of augmenting a target object with projected light is disclosed. The method includes determining a blend of component attributes to define visual characteristics of the target object, modifying an input image based, at least in part, on an image of the target object, wherein the modified input image defines an augmented visual characteristic of the target object, determining a present location of one or more landmarks on the target object based, at least in part, on the image of the target object, predicting a future location of the one or more landmarks, deforming a model of the target object based on the future location of the one or more landmarks, generating an augmentation image based on the deformed model and the modified input image, and transmitting for projection the augmentation image.

Abstract: A method of transferring a facial expression from a subject to a computer-generated character and a system and non-transitory computer-readable medium for the same. The method can include receiving an input image depicting a face of a subject; matching a first facial model to the input image; generating a displacement map representing of finer-scale details not present in the first facial model using a regression function that estimates the shape of the finer-scale details. The displacement map can be combined with the first facial model to create a second facial model that includes the finer-scale details, and the second facial model can be rendered, if desired, to create a computer-generated image of the face of the subject that includes the finer-scale details.

Abstract: Training data from multiple types of sensors and captured in previous capture sessions can be fused within a physics-based tracking framework to train motion priors using different deep learning techniques, such as convolutional neural networks (CNN) and Recurrent Temporal Restricted Boltzmann Machines (RTRBMs). In embodiments employing one or more CNNs, two streams of filters can be used. In those embodiments, one stream of the filters can be used to learn the temporal information and the other stream of the filters can be used to learn spatial information. In embodiments employing one or more RTRBMs, all visible nodes of the RTRBMs can be clamped with values obtained from the training data or data synthesized from the training data. In cases where sensor data is unavailable, the input nodes may be unclamped and the one or more RTRBMs can generate the missing sensor data.

Abstract: Systems and methods are described for leveraging distributed computation systems to split animation application processing of node graphs into two components: 1) a low complexity, primary node graph, that is evaluated by the application of the local device; and 2) one or more, higher complexity, companion node graphs that connect to the primary node graph, and are evaluated by a distributed computation system. As the local device evaluates the original, low complexity node graph, an artist is provided with fast, direct manipulation of an animated object. At the same time, the distributed computation system evaluates the higher complexity, companion node graphs, thereby providing a user of the local application with higher fidelity versions of the primary node graph as they are computed.

Abstract: Enhanced removing of noise and outliers from one or more point sets generated by image-based 3D reconstruction techniques is provided. In accordance with the disclosure, input images and corresponding depth maps can be used to remove pixels that are geometrically and/or photometrically inconsistent with the colored surface implied by the input images. This allows standard surface reconstruction methods (such as Poisson surface reconstruction) to perform less smoothing and thus achieve higher quality surfaces with more features. In some implementations, the enhanced point-cloud noise removal in accordance with the disclosure can include computing per-view depth maps, and detecting and removing noisy points and outliers from each per-view point cloud by checking if points are consistent with the surface implied by the other input views.

Abstract: A recommender engine is configured to access memory and surface transmedia content items; and/or linked transmedia content subsets; and/or one or more identifications of identified users; and/or content items of the plurality of transmedia content items associated with at least one identified user. The surfaced items are presented for selection by the given user via the transmedia content linking engine as one or more user-selected transmedia content items.

Abstract: There is provided a system for linking transmedia content subsets. A memory stores a plurality of transmedia content data items and associated linking data which define time-ordered content links between the plurality of transmedia content data items. The plurality of transmedia content data items are arranged into linked transmedia content subsets comprising different groups of the transmedia content data items and different content links therebetween; a transmedia content model that represents the transmedia content data items as nodes and the content links between the transmedia content data items as edges in one or more time-varying graphs. A processor is configured to associate the transmedia content data items with the time-ordered content links and store the linking data in the memory. It assigns the transmedia content data items to nodes of a graph structure, assign the time-ordered content links to edges of the graph structure and store them in the transmedia content model.

Abstract: A system and method for non-invasive reconstruction of an entire object-specific or person-specific teeth row from just a set of photographs of the mouth region of an object (e.g., an animal) or a person (e.g., an actor or a patient) are provided. A teeth statistic model defining individual teeth in a teeth row can be developed. The teeth statistical model can jointly describe shape and pose variations per tooth, and as well as placement of the individual teeth in the teeth row. In some embodiments, the teeth statistic model can be trained using teeth information from 3D scan data of different sample subjects. The 3D scan data can be used to establish a database of teeth of various shapes and poses. Geometry information regarding the individual teeth can be extracted from the 3D scan data. The teeth statistic model can be trained using the geometry information regarding the individual teeth.

Abstract: A method of rendering content items on a display via an electronic device involves mapping linked content items to a three-dimensional object defined by layout data. The layout data is then transmitted to an electronic device for display.

Abstract: A method is provided for rendering a representation of and interacting with transmedia content on an electronic device. Transmedia content data is received at the electronic device. The transmedia content data comprises: a plurality of transmedia content data items; linking data which define time-ordered content links between the plurality of transmedia content data items, whereby the plurality of transmedia content data items are arranged into linked transmedia content subsets comprising different groups of the transmedia content data items and different content links therebetween; a visualisation model of the transmedia content data; and a hierarchical structure of the linked transmedia content subsets and clusters of linked transmedia content subsets.

Abstract: A system for managing non-linear transmedia content data is provided. Memory stores a plurality of transmedia content data items and associated linking data which define time-ordered content links between the plurality of transmedia content data items. The plurality of transmedia content data items are arranged into linked transmedia content subsets comprising different groups of the transmedia content data items and different content links therebetween. A control engine receives one or more instructions to create a new time-ordered content link between at least two of the plurality of transmedia content data items. The control engine modifies the linking data stored in the memory to include the new time-ordered content link.

Abstract: A distributed computation system is leveraged to evaluate many possible results of a rig near the current pose to obtain predicted poses. A user locally runs an animation application instance to manipulate an animated object including a rig and shape. Depending on a user's selection or modification of a rig control of the rig, changes to the rig control are predicted. Based on the predicted changes to the rig control, the distributed computation system evaluates the rig to obtain different predicted poses. The predicted poses are made available to an interpolator that blends the results to obtain an interpolated shape. In implementations, the distributed computation system may be used to evaluate predicted poses for animation ghosting.

Abstract: One embodiment of the present disclosure may take the form of a projection module for projecting onto a projection surface. The projection module includes a light source emitting a beam of light, a diffractive optical element in optical communication with the light source, the diffractive optical element acts to split the beam of light into a plurality of diffracted light beams, a lens assembly in optical communication with the diffractive optical element and configured to converge the diffracted light beams onto the projection surface, and a motion assembly connected to at least one of the light source or the lens assembly. The motion assembly changes an orientation of the diffractive light beams relative to the lens assembly.

Abstract: Techniques are described for retrieving advertisements for insertion into a document. Embodiments receive a document comprising a request to retrieve a plurality of advertisement creatives. In response to transmitting the request specified within the received document to the advertisement management system, a single software object defining all of the plurality of advertisement creatives is received. Embodiments insert each of the plurality of advertisement creatives into the document, prior to rendering the document, and then render the document together with the advertisement creatives.

Abstract: Systems and methods are described for placing large objects and objects separated by large distances in an AR environment. An AR headset system may place and generate digital objects using relative geographical coordinates (e.g., latitude, longitude, and altitude) between the user's current position and the object being placed. In one implementation, a digital object's geographical coordinates may be calculated by determining a user's geographical coordinates, using a distance determination device to measure a distance to a boundary in the user's real-world environment within an AR headset's field of view, and calculating an orientation of the AR headset relative to the user's position.

Abstract: Embodiments can provide adaptive image filtering. Under this approach, image quality can be enhanced by adjusting an approximation function to better adapt image signals in different parts of an image. Certain parts of the image may be enhanced using a certain approximation function while some other parts of the image may be enhanced using a different approximation function. In certain embodiments, the approximation function selected for a part of the image can be a polynomial function having a specific order. The specific polynomial order can be applied directly to obtain an estimated image value of the part of the image. In certain embodiments, the estimation of the reconstruction error can include iteratively estimating a bias term of the reconstruction error and a variance term of the reconstruction error.

Abstract: The present disclosure provides systems and methods for a mobile device to select content, such as user profiles, contact information, images, and/or videos, to automatically stream to available displays when the mobile device moves within a certain range of the displays. In one embodiment, a method at a mobile device comprises receiving a user input to select a media content asset; detecting a display within a proximity of the mobile device; and sending the selected media content asset to the display without requiring additional user input.

Abstract: Systems and methods for synchronizing movement, such as the movement of a vehicle with an augmented reality (AR) or virtual reality (VR) story-telling or narrative experience is provided. For example, the AR or VR experience can be adapted to justify the movement of the vehicle based on trip criterion, such as a navigational path upon which the vehicle has embarked. Alternatively, trip criterion, such as the navigational path upon which the vehicle will embark, can be adapted to comport with one or more events of the AR or VR experience.

Abstract: An embodiment provides a motorized monopole for a camera, including: a hand-held monopole; a first motor positioned at an end of the hand-held monopole; a first connecting element attached to the first motor; a second motor positioned at an end of the first connecting element; a second connecting element attached to the second motor; a third motor positioned at an end of the second connecting element; and a camera mounting plate attached to the second connecting element by the third motor, where components of the multi-axis gimbal are positioned such that a camera viewing axis, a horizontal image axis, and a vertical image axis of a camera mounted on the camera mounting plate need not be aligned with any of a rotational axis of the first motor, a rotational axis of the second motor, or a rotational axis of the third motor. Other aspects are described and claimed.

Abstract: Systems and methods are provided that allow a display device array to be used to present content to one or more users. In particular, a user associated with a first user device, may opt to present content on a plurality of other devices. The plurality of other devices may be arranged in any desired configuration, and the first user device can capture the arrangement. Upon capturing the arrangement, content, under the control of the first user device, may be presented on the plurality of other devices in accordance with the arrangement.

Abstract: Systems and techniques for generating a parametric eye model of one or more eyes are provided. The systems and techniques may include obtaining eye data from an eye model database. The eye data includes eyeball data and iris data corresponding to a plurality of eyes. The systems and techniques may further include generating an eyeball model using the eyeball data. Generating the eyeball model includes establishing correspondences among the plurality of eyes. The systems and techniques may further include generating an iris model using the iris data. Generating the iris model includes sampling one or more patches of one or more of the plurality of eyes using an iris control map and merging the one or more patches into a synthesized texture. The systems and techniques may further include generating the parametric eye model that includes the eyeball model and the iris model.

Abstract: Systems and techniques for reconstructing one or more eyes using a parametric eye model are provided. The systems and techniques may include obtaining one or more input images that include at least one eye. The systems and techniques may further include obtaining a parametric eye model including an eyeball model and an iris model. The systems and techniques may further include determining parameters of the parametric eye model from the one or more input images. The parameters can be determined to fit the parametric eye model to the at least one eye in the one or more input images. The parameters include a control map used by the iris model to synthesize an iris of the at least one eye. The systems and techniques may further include reconstructing the at least one eye using the parametric eye model with the determined parameters.

Abstract: Systems, devices, and methods are disclosed for providing co-deliver of primary and secondary content, including, video and video chat content. A first electronic device includes a transceiver that receives the secondary content from a second electronic device. The first electronic device also includes components capable of outputting aspects of the primary and secondary content in a synchronized fashion. Further, the first electronic device includes circuitry coupled to the transceiver and components, and the circuitry executes instructions in order to perform functions. One such function is to synchronize local primary content to be output by the components, with delivered primary content to be provided by the second electronic device. Another such function is to signal the components to output aspects of the local primary or the secondary content.

Abstract: A method and system for simulation of deformation of a thin-shelled member are disclosed herein. The method includes: receiving at one or more computer systems, information identifying a computer-generated object. The computer-generated object can be a thin-shelled member. The method includes: receiving information identifying a discretization of the computer-generated object, which discretization can be a plurality of nodes; receiving information identifying a set of material properties for the computer-generated object; pre-filtering nodes from the discretization based on predicted collisions; generating a preconditioner via a preconditioning algorithm; and iteratively solving for nodes at a plurality of time points via a conjugate gradient method.

Abstract: Systems and methods for presenting augmented reality (AR) or virtual reality (VR) representations of digital content entitlements owned by a user are provided. An AR or VR environment is generated in which the AR or VR representations are presented to simulate a physical collection of media. The user is allowed to interact with the AR or VR representations, thereby triggering one or more actions, such as presentation of the corresponding digital content, presentation of additional, related content, presentation of an option to engage in a purchase transaction for related content or products, etc.

Abstract: Systems and methods for generating and outputting personalized content recommendations are provided. When it is determined that two or more users are in close proximity to each other, the two or more users' respective media content libraries may be compared. Commonalities or related aspects of the two or more users' respective media content libraries can be determined and leveraged to generated personalized content recommendations.

Abstract: Systems and methods are provided for generating high dynamic range (HDR) content using existing standard dynamic range (SDR) content. HDR content is generated by restoring lost detail in the SDR content using source content from which the SDR content was derived. The HDR content generated from the SDR content also preserves any color characteristics, such as color grading, of the SDR content in conjunction with the restoration of lost detail.

Abstract: Systems, devices, and methods are disclosed for providing co-deliver of primary and secondary content, including, video and video chat content. A first electronic device includes a transceiver that receives the secondary content from a second electronic device. The first electronic device also includes components capable of outputting aspects of the primary and secondary content in a synchronized fashion. Further, the first electronic device includes circuitry coupled to the transceiver and components, and the circuitry executes instructions in order to perform functions. One such function is to synchronize local primary content to be output by the components, with delivered primary content to be provided by the second electronic device. Another such function is to signal the components to output aspects of the local primary or the secondary content.

Abstract: One particular implementation of the present invention may take the form of a method and system for encoding decoding 3-D multimedia content. In one example, visual content having a plurality of source frames may be processed to create a plurality of first and second type frames from each of the plurality of source frames. The plurality of first and second type frames may be arranged to create modified content, and generating a distinguishing signal for the visual content, which may distinguish the first type frame from the second type frame. In another examples, the present invention may take the form of a method decoding 3-D multimedia content encoded with at least content frames and null frames. The method may operate to receive the 3-D multimedia content, extract at least one content frame, at least one null frame, and display information.

Abstract: Systems and methods are described for linking and interacting between augmented reality (AR) and virtual reality (VR) environments. An AR host system may capture or retrieve data associated with an AR view of a first user's real-world environment. The data may include an audio-visual stream of the real-world view, digital object data associated with digital objects overlaid on the real-world view, and other data. The AR host system transmits the data to a VR system. The VR system may place a second user into a VR environment that is a representation of the AR view. The VR system may also generate additional digital object data associated with the AR view and transmit the generated digital object data to the AR host system. The AR host system receives the digital object data and modifies the AR view presented to the first user.

Abstract: Systems and methods are provided for proximity-based sorting. Information may be transmitted to user devices from beacons or similar transmitter-type devices. Based on this information, the user devices may be categorized or sorted based upon which beacons the user devices are proximate to, pass, or from which beacon the information is received. Subsequent activities and/or operations may then leverage this categorization or sorting of users.

Abstract: Systems and methods for re-arranging a musical composition using augmented reality are disclosed. A user can be provided with one or more physical image markers representative of at least one of a musical entity and a musical style in which a musical track has been pre-recorded. Upon capturing a visual scene in which one or more of these physical image markers are included, an augmented reality version of the visual scene is displayed along with audio corresponding to the musical style(s) and/or entities represented by the one or more physical image markers.

Abstract: Systems and methods are described for leveraging machine learning to deliver time-sensitive media content offers to a consumer over an optimal device and channel. In an implementation, a server system transmits media content offers over an optimal device and channel to a user. The server system captures metadata of at least one of: the user's use of multiple devices, the user's media content preferences, the user's preferred transmission channels for receiving media content offers over the multiple devices, and the user's preferred times for receiving media content offers. The server system prepares a media content offer for transmission to the user. Based on the captured metadata, the server system selects a user device of the multiple user devices and a transmission channel of multiple transmission channels to transmit the media content offer; and transmits the media content offer to the selected user device over the selected transmission channel.

Abstract: A system for identifying, tagging, and displaying one or more assets within an audio visual (AV) file includes an asset tagging server, an asset tag acquisition device, and a database, wherein the asset tag acquisition device includes an asset identification engine configured to receive an asset identification data set and generate an asset tag data file that includes an asset tag corresponding to each asset. The asset tagging server is configured to receive, from an AV capture device, an AV file that includes an AV representation of each asset and corresponding timestamp data, and store, in the database, an AV asset tag data file comprising the beginning timestamp, the ending time stamp, and the set of asset tags.

Abstract: Methods and systems of dynamic content analysis for categorizing media contents are provided. Data regarding the media contents that a user is watching on one or more devices is obtained, and the environment in which the user is watching the media contents may be captured. Metadata categorizing the events and the media contents can be generated, and the metadata may be further associated with a corresponding event.

Abstract: Techniques and systems are described for performing video segmentation using fully connected object proposals. For example, a number of object proposals for a video sequence are generated. A pruning step can be performed to retain high quality proposals that have sufficient discriminative power. A classifier can be used to provide a rough classification and subsampling of the data to reduce the size of the proposal space, while preserving a large pool of candidate proposals. A final labeling of the candidate proposals can then be determined, such as a foreground or background designation for each object proposal, by solving for a posteriori probability of a fully connected conditional random field, over which an energy function can be defined and minimized.