Abstract: The disclosure is directed to providing operator visibility through an object that occludes the view of the operator by using a HMD system in communication with one or more imaging devices. A method of doing so may include: using one or more imaging devices coupled to an exterior of an object to capture image data of a real-world environment surrounding the object: calculating an orientation of a HMD positioned in an interior of the object; using at least the calculated orientation of the HMD, synthesizing a FOV of the image data with a FOV of the HMD; and rendering the synthesized FOV of the image data to the HMD. In some instances, the captured image data is used to create a three-dimensional model of the real-world environment, and synthesizing the FOV of the image data includes synthesizing a FOV of the 3D model with the FOV of the HMD.

Abstract: Systems and methods for correcting color of uncalibrated material is disclosed. Example embodiments include a system to correct color of uncalibrated material. The system may include a non-transitory computer-readable medium operatively coupled to processors. The non-transitory computer-readable medium may store instructions that, when executed cause the processors to perform a number of operations. One operation is to obtain a target image of a degraded target material with one or more objects. The degraded target material comprises degraded colors and light information corresponding to light sources in the degraded target material. Another operations is to obtain color reference data. Another operation is to identify an object in the target image that corresponds to the color reference data. Yet another operation is to correct the identified object in the target image. Another operation is to correct the target image.

Abstract: A method and system provide the ability to orchestrate a media workflow using a computer system. A payload for an application programming interface (API) to be utilized to process media content is determined. The payload is parsed to display one or more variables of the payload in a graphical user interface. The variables are mapped to webform user interface elements. A job of the media workflow is executed. During execution, a webform of the user interface elements is programmatically generated and presented to an operator, data for the payload is received via the webform, and the received data is provided to the API to process the media content.

Abstract: Systems and methods for providing machine-learning enabled user-specific evaluations are disclosed. Implementations include obtaining a first set of evaluation data from a user interface, obtaining a first set of target-descriptive data including target-specific characteristics objectively describing the evaluation targets, and training, with a machine-learning algorithm, a user-specific evaluation profile indicating evaluation patterns relative to the first set of evaluation data and the first set of target-specific characteristics. Implementations include applying the user-specific evaluation profile to a second set of target-descriptive data to predict a user-specific evaluation.

Abstract: Systems, devices, and methods are disclosed for compositing real-time, real-world video with virtual objects. An electronic device includes circuitry coupled to a memory storing instructions that, when executed, cause the circuitry to receive video of a given video capture region. The circuitry is caused to receive location information and camera information from the unmanned vehicle. The circuitry is caused to obtain a representation of a location of interest corresponding to the location of the unmanned vehicle. The circuitry is caused to display the representation. The circuitry is caused to obtain virtual objects. The circuitry is caused to place the virtual objects into the representation. The circuitry is caused to render the virtual objects and the video to generate a rendered video including rendered virtual objects in at least one frame of the video. The circuitry is caused to send the rendered video.

Abstract: Systems and methods are provided for providing a platform to provide virtual storefronts to consumers. Environmental elements are associated with specific consumer services on computer server. A user in the environment takes audio or visual recordings of an environmental element and uploads the recordings to the server. The server determines the appropriate consumer service associated with the recorded environmental element and provides the user with a reference to the service. Accordingly, any space, such as a bus stop, subway or train terminal, movie theater, or airport, may be turned into a virtual store.

Abstract: A system for identifying, tagging, and displaying one or more assets within an audio visual (AV) file includes an asset tagging server, an asset tag acquisition device, and a database, wherein the asset tag acquisition device includes an asset identification engine configured to receive an asset identification data set and generate an asset tag data file that includes an asset tag corresponding to each asset. The asset tagging server is configured to receive, from an AV capture device, an AV file that includes an AV representation of each asset and corresponding timestamp data, and store, in the database, an AV asset tag data file comprising the beginning timestamp, the ending time stamp, and the set of asset tags.

Abstract: Systems and methods for distortion removal at multiple quality levels are disclosed. In one embodiment, a method may include receiving training content. The training content may include original content, reconstructed content, and training distortion quality levels corresponding to the reconstructed content. The reconstructed content may be derived from distorted original content. The method may also include training distortion quality levels corresponding to the reconstructed content. The method may further include receiving an initial distortion removal model. The method may include generating a conditioned distortion removal model by training the initial distortion removal model using the training content. The method may further include storing the conditioned distortion removal model.

Abstract: Systems and methods for an interactive communications system capable of generating a response to conversational input are provided. The interactive communications system analyzes the conversational input to determine relevant topics of discussion. The interactive communications system further determines which of the relevant topics of discussion can potentially lead to an unwanted end to a conversation. The interactive communications system redirects the conversation by providing responses to the conversational input that are intended simply to avoid the unwanted end to the conversation.

Abstract: The disclosure is directed toward stabilizing an image projected by a projector in an environment including vibrations or other movements that displace the projected image. In one set of implementations, an image projected by a projector may be stabilized by using an image stabilization system that detects a displacement of the projector from an equilibrium position and controls an optical element of the projector to offset the displacement. In an additional set of implementations, an image projected by the projector may be stabilized by using an image stabilization system that measures a rate of displacement from equilibrium of an image projected by the projector to predict a displacement of the projector and counteract the predicted displacement. In another set of implementations, an image projected by a projector may be stabilized by precalculating a plurality of lookup tables that correct for displacements of the projected image during operation of the projector.

Abstract: A modular architecture is provided for denoising Monte Carlo renderings using neural networks. The temporal approach extracts and combines feature representations from neighboring frames rather than building a temporal context using recurrent connections. A multiscale architecture includes separate single-frame or temporal denoising modules for individual scales, and one or more scale compositor neural networks configured to adaptively blend individual scales. An error-predicting module is configured to produce adaptive sampling maps for a renderer to achieve more uniform residual noise distribution. An asymmetric loss function may be used for training the neural networks, which can provide control over the variance-bias trade-off during denoising.

Abstract: A modular architecture is provided for denoising Monte Carlo renderings using neural networks. The temporal approach extracts and combines feature representations from neighboring frames rather than building a temporal context using recurrent connections. A multiscale architecture includes separate single-frame or temporal denoising modules for individual scales, and one or more scale compositor neural networks configured to adaptively blend individual scales. An error-predicting module is configured to produce adaptive sampling maps for a renderer to achieve more uniform residual noise distribution. An asymmetric loss function may be used for training the neural networks, which can provide control over the variance-bias trade-off during denoising.

Abstract: A modular architecture is provided for denoising Monte Carlo renderings using neural networks. The temporal approach extracts and combines feature representations from neighboring frames rather than building a temporal context using recurrent connections. A multiscale architecture includes separate single-frame or temporal denoising modules for individual scales, and one or more scale compositor neural networks configured to adaptively blend individual scales. An error-predicting module is configured to produce adaptive sampling maps for a renderer to achieve more uniform residual noise distribution. An asymmetric loss function may be used for training the neural networks, which can provide control over the variance-bias trade-off during denoising.

Abstract: Systems and methods for displaying object features via an AR device are disclosed. The method may include receiving, from a transmitter, a signal from an object on a movie set. The signal may specify object data. The object may be a prop. The object data may specify one or more features corresponding to the prop. The method may include generating, with the one or more physical computer processors and the one or more AR components, a representation of the one or more features using visual effects to depict at least some of the object data. The method may include displaying, via the AR device, the representation onto a view of the prop.

Abstract: Systems and methods for tracking objects in a field of view are disclosed. In one embodiment a method may include obtaining, from the non-transient electronic storage, object data. The object data may include a position of one or more objects as a function of time in a field of view. The method may include generating, with the one or more physical computer processors and the one or more AR components, a first virtual object to depict at least one or more of the object data of a first object at a first time. The method may include displaying, via the display, the first virtual object.

Abstract: The disclosure provides for a system of markers and methods of using the system of markers to provide a precise scene scale reference for captured aerial images. Each of the markers may include one or more pairs of aligned collimated light emitters, where each pair of light emitters is configured to emit two light beams that converge at a known distance from the marker. When two or more markers are used, the system of markers may be aligned in a unique physical orientation to form a shape of known dimensions (e.g., a line, a triangle, or square) that provides an accurate scene scale reference for captured images.

Abstract: Systems and methods are described for dynamically adjusting an amount of retrieved recognition data based on the needs of a show, experience, or other event where participants are recognized. The retrieved recognition data may be deleted once it is no longer needed for the event. Recognition data retrieval is limited to just what is needed for the particular task, minimizing the uniqueness of any retrieved recognition data to respect participant privacy while providing an enhanced participant experience through recognition.

Abstract: An air flow generator may be implemented on an augmented reality (AR) or virtual reality (VR) controller or head-mounted display (HMD) through which an AR or VR experience is presented. Based on content upon which the AR or VR experience is based, air flow effects can be provided by the air flow generator. In particular, desired air flow effect parameters based on or obtained from the content, can be used to enhance the AR or VR experience through generating air flow directed at a user of the HMD. The air flow generated by the air flow generator can be further enhanced by the addition of liquid and/or scented additives.

Abstract: Systems, devices, and methods are disclosed for generating improved AR content. An electronic device includes circuitry coupled to a memory storing instructions that, when executed, cause the circuitry to obtain frame data for a frame captured using a camera. The frame data includes a level of focus for one or more frame objects in the frame. The circuitry is caused to associate an augmented reality object with at least one of the one or more frame objects. The circuitry is caused to determine a fit factor between a level of focus of the augmented reality object and level of focus of the at least one of the frame objects associated with the augmented reality object. Additionally, the circuitry is caused to, if the fit factor does not satisfy a threshold, apply a decrease to the level of focus of the augmented reality object in order to generate an increased fit factor.

Abstract: Systems and methods for a computer-based, interactive communications system capable of generating a response to a human language input are provided. The computer-based, interactive communications systems includes a plurality of response models that may be selected to process one or more keywords extracted from the human language input. The plurality of response models may include at least one conversational response model and at least one informational response model, so that the computer-based, interactive communications system is able to respond to the human language input in a manner commensurate with the type of human language input it receives.

Abstract: Systems and methods for selectively enabling enhanced high dynamic range (HDR) metadata for enhanced HDR playback of media content stored on a media disc are provided. Media discs may be authored with a base layer media content in accordance with a non-enhanced HDR media profile. Media discs may also include enhanced HDR metadata. In order to enable the use of the enhanced HDR metadata, a media player in which the media disc is inserted for playback may first seek authorization from a third-party resource, such as an authorization server. Only after obtaining authorization is the media player allowed to playback the media content stored on the media disc in conjunction with the enhanced HDR metadata. If authorization is not obtained, the media player may only play the base layer media content.

Abstract: Systems and methods are provided for manually or automatically identifying delineating scenes in media content. Scene categorization parameters setting forth thematic and/or cinematic criteria are used to analyze frames or groups of pictures (GOPs) making up the media content. Those frames or GOPs having one or more characteristics that meet the thematic and/or cinematic criteria may be deemed to belong or be a part of a particular scene in the media content. Downstream applications or processes can be optimized based on the scene segmentation.

Abstract: Systems and methods are provided for transcoding media content in accordance with thematic and/or cinematic characteristics of scenes. In particular, compression techniques may be selected to optimize compression of scenes based on their thematic and/cinematic characteristics. Scene boundary identifiers, such as scene in and out points may be identified, and the corresponding thematic and/or cinematic characteristic meta-data can be sent to a transcoder for transcoding. In this way, the specific characteristics of each scene in a movie can be accounted for during transcoding. This can result in transcoded versions of the movie that have better audio/visual qualities, and that can be encoded at a lower overall bit rate.

Abstract: Systems and methods are provided for a workflow framework that scriptwriters can utilize when developing scripts. A script can be parsed to identify one or more elements in a script, and various visual representations of the one or more elements and/or a scene characterized in the script can be automatically generated. A user may develop or edit the script which can be presented in a visual and temporal manner. Information parsed from the script can be stored in basic information elements, and used to create a knowledge bases.

Abstract: The present disclosure is directed to presenting a more realistic augmented reality view on a video see-through display of a device by configuring the device such that the displayed image of the real world substantially matches what would be perceived by the user if the display were not present. This may be implemented by determining one or more of: a distance from the user's eyes to the display of the device, and an angular offset between the optical axis of a rear camera of the device and the user's visual field, and using the determined distance and/or angular offset to adjust the image that is displayed to the user. The image that is displayed to the user may be adjusted by optically or digitally zooming the rear camera of the device. It may also be adjusted by tilting the rear camera or by digitally translating the displayed video feed.

Abstract: Implementations of the disclosure describe systems and methods for a user that interacts with an interface for playback/consumption of media content (e.g. video, audio, e-book text, etc.) to be accurately guided back to one or more positions on a scrubber. Current scrubber positions (e.g., timecodes) may be dynamically saved each time that a user interacts with the scrubber. Thereafter, the saved position may be used to help guide the user back to their previous position along the scrubber. In some implementations, a scrubber may snap back to a saved position if it is within a predetermined threshold distance of the saved position. In some implementations, sensory cues such as haptic, visual, and/or auditory feedback may be provided when the scrubber snaps back to a saved position.

Abstract: Systems and methods are provided for a workflow framework that scriptwriters can utilize when developing scripts. A script can be parsed to identify one or more elements in a script, and various visual representations of the one or more elements and/or a scene characterized in the script can be automatically generated. A user may develop or edit the script which can be presented in a visual and temporal manner. Information parsed from the script can be stored in basic information elements, and used to create a knowledge bases.

Abstract: The invention relates to systems and methods for manipulating non-linearly connected transmedia content, in particular for creating, processing and/or managing non-linearly connected transmedia content and for tracking content creation and attributing transmedia content to one or more creators. Specifically, the invention involves creating a transmedia content data item by a first user and storing the transmedia content data item in a data store, along with a record indicating an association between the first user and the transmedia content data item; creating an ordered group of transmedia content data items by a second user, the ordered group comprising a pointer to the transmedia content data item of the first user; and storing the ordered group and a record associating both the first user and the second user with the ordered group in the data store.

Abstract: The present disclosure relates to a method for calibrating a projector. In one example, the method includes receiving by a processing element light field data corresponding to a calibration image projected by a projector and captured by a light field capturing device, and modeling by a processing element one or more intrinsic properties of the projector using the light field data and the calibration image. The calibration image may be projected by the projector directly into the light field capturing device.

Abstract: The present invention relates to systems and methods for manipulating non-linearly connected transmedia content, in particular for creating, processing and managing non-linearly connected transmedia content. The disclosure includes content creation modes and version control systems for non-linearly connected transmedia content data, in which individual transmedia content data items are connected to either other via directional linking data.

Abstract: The invention relates to systems and methods for navigating, outputting and displaying non-linearly connected groups of transmedia content. Specifically, the invention involves retrieving, from a database, an ordered group of transmedia content data objects comprising a plurality of transmedia content data objects and linking data, whereby each element of the linking data defines a directional link from one of the transmedia content data objects to another of the transmedia content data objects; generating a two-dimensional graph structure representing the ordered group of transmedia content data objects whereby each graph node corresponds to a transmedia content data object and each graph edge corresponds to an element of the linking data; and outputting the graph structure on a display screen.

Abstract: The present disclosure relates to an apparatus, system and method for processing transmedia content data. More specifically, the disclosure provides for identifying and inserting one item of media content within another item of media content, e.g. inserting a video within a video, such that the first item of media content appears as part of the second item. The invention involves analysing a first visual media item to identify one or more spatial locations to insert the second visual media item within the image data of the first visual media item, detecting characteristics of the one or more identified spatial locations, transforming the second visual media item according to the detected characteristics and combining the first visual media item and second visual media item by inserting the transformed second visual media item into the first visual media item at the one or more identified spatial locations.

Abstract: An apparatus such as a head-mounted display (HMD) may have a camera for capturing a visual scene for presentation via the HMD. A user of the apparatus may specify a pre-planned travel route for a vehicle within the visual scene via an augmented reality (AR) experience generated by the HMD. The pre-planned travel route may be overlaid on the visual scene in the AR experience so that the user can account for real-time environmental conditions determined through the AR experience. The pre-planned travel route may be transferred to the vehicle and used as autonomous travel instructions.

Abstract: Systems and methods described herein are directed to enhancing a virtual reality (VR) or augmented reality (AR) experience by using one or more unmanned vehicles to generate effects around a user of a headmounted display (HMD). The generated effects may be synchronized with VR/AR content presented to the user of the HMD. Particular systems and methods described herein are directed to enhancing a VR/AR experience by using one or more unmanned aerial vehicles (UAV) to generate air flow effects around a user of a HMD. The air flow effects generated by UAV may simulate a physical and/or olfactory sensation corresponding to a VR/AR environment presented to the user using the HMD.

Abstract: The disclosure is directed to collaborative multi-modal mixed-reality systems and methods leveraging reconfigurable tangible user interfaces for the production of immersive, cinematic, and interactive content. As described herein, content may be created as either a mixture of live action media and computer generated media, combined together in context; or content made up entirely from computer-generated media alone. The technology described herein may leverage physical and software-based user interfaces that merge and unite the physical world and the virtual world, which allow users to produce new forms of content with a user experience and user interactions that emulate traditional processes.

Abstract: Techniques described herein are directed to adaptive virtual reality and augmented reality viewing based on a user's eye condition data. In a first implementation, a software application renders video content based on the user's eye condition data by mapping the user's eye condition data to video rendering parameters. The video content rendered based on the user's eye condition data may be made available to a virtual reality/augmented reality player and played using a head mounted display. The video content rendered based on the user's eye condition data may be pre-rendered before video content playback or rendered in real time during video content playback. In an additional implementation, the head mounted display may be calibrated and adjusted based on the eye condition data prior to video playback.

Abstract: Systems and methods described herein are directed to capturing intrinsic color images of subjects. A camera may be equipped with a light source that is coaxial to the camera's image sensor and configured to emit a pulse of light of short duration. During image capture of a subject, the camera light source may emit the pulse of light through the lens barrel of the camera and stop emission of light before the reflected light from the light source returns. Thereafter, the camera lens receives the reflected light from the light source (with the light source no longer emitting light) and charge is collected at one or more image sensor photodetector sites of the camera.

Abstract: The present disclosure provides for systems and methods for predictive delivery of high bit-rate content. The disclosed systems and methods provide an adaptive-bit-rate streaming (ABS) system with more robust information, thereby allowing more intelligent pre-caching of the media content. By providing greater information to the ABS system, the disclosed systems are able to foresee higher bit-rate segments that require greater attention, allowing the system to use such information to improve the playback experience.

Abstract: Systems and methods are provided for reordering and/or bypassing certain informational content or menus that are conventionally presented prior to playback of media content stored on physical media discs. Upon initial use of a physical media disc, certain information content or menus may be presented to a user or viewer, for example, piracy warnings, language selection menus, etc. However, upon subsequent use of the physical media disc, such informational content or menus may be bypassed. The user or viewer is given an option to immediately begin consuming the media content stored on the physical media disc. Conventional content, such as trailers are not played prior to playback of the media content.

Abstract: Systems and methods for efficiently distributing digital cinema files from a server to a theater are disclosed. The systems and methods may comply with DCI requirements. A server updates an original composition playlist, updates a track file associated with the updated composition playlist, and distributes the updated composition playlist and updated track file to one or more theaters over a communications network. The theater is configured to store the updated track file at a storage containing copies of all track files associated with the updated composition playlist and update the theater's original composition playlist to the updated composition playlist. The server may have a database configured to track the composition playlists and associated track files present in each theater in the system.

Abstract: Disclosed are systems and methods for user interaction with and curation of digital media content, such that users are able to specify a particular clip of video content, and utilize the clip in a desired way. The disclosed systems and methods allow users to view video content, select video clips within the video content, save video clips into a collection of video clips, and curate the collection of video clips. The disclosed systems and methods also allow users to view bookmarks indicating the positions of saved video clips within video content, and to interact with video clips within a video player.

Abstract: Systems and methods for providing physical navigation guidance via augmented reality (AR) environments are provided. An AR experience is generated based on navigation information including a starting location and a destination location. The AR experience comprises physical navigation guidance from the starting location to the destination location using virtual elements that are associated with either or both the starting location and/or the destination location. For example, virtual elements in the AR environment can transition from virtual elements associated with the starting location to virtual elements associated with the destination location as the user progresses from the starting location to the destination location.

Abstract: Features of the surface of an object of interest captured in a two-dimensional (2D) image are identified and marked for use in point matching to align multiple 2D images and generating a point cloud representative of the surface of the object in a photogrammetry process. The features which represent actual surface features of the object may have their local contrast enhanced to facilitate their identification. Reflections on the surface of the object are suppressed by correlating such reflections with, e.g., light sources, not associated with the object of interest so that during photogrammetry, such reflections can be ignored, resulting in the creation of a 3D model that is an accurate representation of the object of interest. Prior to local contrast enhancement and the suppression of reflection information, identification and isolation of the object of interest can be improved through one or more filtering processes.

Abstract: Systems for electronic media distribution includes a differential versioning server configured to receive a first media file including a first set of data with a first set of attributes and a second media file including a second set of data with a second set of attributes, generate a first differential data file as a function of differences between the first media file and the second media file, and generate a first differential metadata file including an encoding data set configured to enable a media decoder to regenerate the second media file by applying the first differential data file to the first media file. Systems for electronic media distribution may also include a receiver communicatively coupled to a differential versioning decoder configured to receive the first media file and the first differential data file and generate the second media file by applying the first differential data file to the first media file.

Abstract: Enhanced removing of noise and outliers from one or more point sets generated by image-based 3D reconstruction techniques is provided. In accordance with the disclosure, input images and corresponding depth maps can be used to remove pixels that are geometrically and/or photometrically inconsistent with the colored surface implied by the input images. This allows standard surface reconstruction methods (such as Poisson surface reconstruction) to perform less smoothing and thus achieve higher quality surfaces with more features. In some implementations, the enhanced point-cloud noise removal in accordance with the disclosure can include computing per-view depth maps, and detecting and removing noisy points and outliers from each per-view point cloud by checking if points are consistent with the surface implied by the other input views.

Abstract: Systems and methods are provided for a workflow framework that scriptwriters can utilize when developing (live-action/animation/cinematic) virtual reality (VR) experiences or content. A script can be parsed to identify one or more elements in a script, and a VR representation of the one or more elements can be automatically generated. A user may develop or edit the script which can be presented in a visual and temporal manner along with the VR representation. The user may edit the VR representation, and the visual and temporal presentation of the script can be commensurately represented. The script may be analyzed for consistency and/or cohesiveness in the context of the VR representation or experience. A preview of the VR experience or content can be generated from the script and/or the VR representation.

Abstract: Supervised machine learning using convolutional neural network (CNN) is applied to denoising images rendered by MC path tracing. The input image data may include pixel color and its variance, as well as a set of auxiliary buffers that encode scene information (e.g., surface normal, albedo, depth, and their corresponding variances). In some embodiments, a CNN directly predicts the final denoised pixel value as a highly non-linear combination of the input features. In some other embodiments, a kernel-prediction neural network uses a CNN to estimate the local weighting kernels, which are used to compute each denoised pixel from its neighbors. In some embodiments, the input image can be decomposed into diffuse and specular components. The diffuse and specular components are then independently preprocessed, filtered, and postprocessed, before recombining them to obtain a final denoised image.

Abstract: Systems and methods for using graph databases to make digital content recommendations are described. A graph database may be associated with tagged digital content. The graph database may include a node for each content tag and edges identifying a relationship between nodes. When a user accesses or searches a digital content item, the graph database may be traversed to identify and present related content recommendations to the user based on the traversed nodes. Node graph traversal may initiate from a root node associated with a tag of content item being presented to a user. Nodes may be searched based on their proximity to the root node as determined by edge connections of the graph. Initially searched nodes may include nodes in the graph database directly connected by edges to the root node. Subsequently searched nodes may include nodes one edge connected removed from the root node, and so forth.

Abstract: Embodiments can provide a strategy for controlling information flow both from known opacity regions to unknown regions, as well as within the unknown region itself. This strategy is formulated through the use and refinement of various affinity definitions. As a result of this strategy, a final linear system can be obtained, which can be solved in closed form. One embodiment pertains to identifying opacity information flows. The opacity information flow may include one or more of flows from pixels in the image that have similar colors to a target pixel, flows from pixels in the foreground and background to the target pixel, flows from pixels in the unknown opacity region in the image to the target pixel, flows from pixels immediately surrounding the target pixels in the image to the target pixel, and any other flow.

Abstract: One or more levels of maladaptation are calculated relative to frames of media content having abrupt jumps from periods of low illumination to bright illumination when visual acuity may be lost and/or discomfort may be experienced. These levels of maladaptation may be correlated with subjectively determined levels of perceived luminance discomfort. Based upon the levels of perceived luminance discomfort that can be derived from the levels of maladaptation, the media content may be adjusted.