Abstract: Improved media processing techniques includes media data formats with a “convenience” sample having both a snapshot and a delta description of a scene at the same media time. In other examples, improved media processing techniques include creating a temporary media sample having a merged snapshot or a merged delta in a context of a pre-render thread to improve performance of rendering performed in a context of a separate rendering thread.

Abstract: Various implementations disclosed herein include devices, systems, and methods that adjust a brightness characteristic of virtual content (e.g., virtual objects) and/or real content (e.g., passthrough video) in views of an XR environment provided by a head mounted device (HMD). The brightness characteristic may be adjusted based on determining a viewing state (e.g., a user's eye perception/adaptation state). A viewing state, such as a user's eye perception/adaptation state while viewing a view of an XR environment via an HMD, may respond to a brightness characteristic of the XR environment that the user is seeing, which is not necessarily the brightness characteristic of the physical environment upon which the view is wholly or partially based.

Abstract: Improved media processing techniques includes media data formats with a “convenience” sample having both a snapshot and a delta description of a scene at the same media time. In other examples, improved media processing techniques include creating a temporary media sample having a merged snapshot or a merged delta in a context of a pre-render thread to improve performance of rendering performed in a context of a separate rendering thread.

Abstract: Techniques are disclosed for identifying multiple sections from one or more tracks of a media file and reading them together in a consumption-driven pipeline process. A render pipeline may comprise a sample generator, a sample buffer, and a destination buffer. Multiple render pipelines may be used for parsing multiple tracks of the media file. An I/O manager may determine that a destination buffer requires new data. The I/O manager may schedule a memory read for a data element from the sample buffer corresponding to the destination buffer and may determine if any of the sample buffers have data elements with memory locations close to the scheduled read. If so, the I/O manager may also schedule those memory locations to be read. After reading, the filled data elements corresponding to the read memory may then be sent to their corresponding destination buffers to be consumed and added to their corresponding tracks.

Abstract: In some implementations, a computing device can notify a media application when a screen recording may be in progress. For example, the computing device can be configured to send media content provided by the media application for presentation on the display of the computing device to a second destination (e.g., a file, a remote playback device, a remote recording device, etc.) that may result in a recording of the display screen of the computing device. To allow the media application an opportunity to protect the media content from unauthorized recording, the computing device can notify the media application that the media content being sent to the second destination may be recorded. In response to receiving the notification, the media application can modify the presentation of the media content to prevent or downgrade any recording that may be performed.

Abstract: Techniques are disclosed for identifying multiple sections from one or more tracks of a media file and reading them together in a consumption-driven pipeline process. A render pipeline may comprise a sample generator, a sample buffer, and a destination buffer. Multiple render pipelines may be used for parsing multiple tracks of the media file. An I/O manager may determine that a destination buffer requires new data. The I/O manager may schedule a memory read for a data element from the sample buffer corresponding to the destination buffer and may determine if any of the sample buffers have data elements with memory locations close to the scheduled read. If so, the I/O manager may also schedule those memory locations to be read. After reading, the filled data elements corresponding to the read memory may then be sent to their corresponding destination buffers to be consumed and added to their corresponding tracks.

Abstract: Techniques are disclosed for selecting frames for decode and display during different playback modes of a media player. Prediction dependencies may be estimated among frames from a sample table of a media item identifying dependency state among frames in the media item. Based on a playback rate of a media player, a collection of frames may be identified from the media item that have presentation times within a display refresh time of the player. A frame may be selected for decode and display during the display refresh time based on the estimated prediction dependencies. The selected frame may be decoded for display during the player display refresh time.

Abstract: Techniques are disclosed for selecting frames for decode and display during different playback modes of a media player. Prediction dependencies may be estimated among frames from a sample table of a media item identifying dependency state among frames in the media item. Based on a playback rate of a media player, a collection of frames may be identified from the media item that have presentation times within a display refresh time of the player. A frame may be selected for decode and display during the display refresh time based on the estimated prediction dependencies. The selected frame may be decoded for display during the player display refresh time.

Abstract: Video data with high frame rates may be displayed on devices with limited resources (e.g., decoder and/or display resources). These devices may have their resources devoted to other tasks or may not be capable to display the video data at the high frame rates. The coding method may include coding the frames such that additional droppable frames are included in the encoded video data. The decoding method may include dropping droppable frames before the encoded video data is decoded to reduce the number of frames that will be decoded and displayed. These methods may be applied to video data that has a variable frame rate and may be combined with processing the image sequence for slow motion playback.

Abstract: Disclosed herein are systems, methods, and non-transitory computer-readable storage media for creating and using media asset proxies. The media asset proxies represent a digital media asset and are created by filtering and modifying elements from the digital media asset. The media asset proxies can be queried in the same manner as their corresponding digital media asset.

Abstract: Methods, systems and machine readable storage medium for allowing playback of streaming media at playback rates of other than 1× are described. In one embodiment, a method can determine different sets of I-frames that are available before a display deadline, where each set can have a different cadence, and one of the sets can be selected for download and display to achieve playback at other than 1×. Byte range requests from a client device can be used to retrieve each of the I-frames. Other methods, system and media are also described.

Abstract: A module may provide codec-independent services including determining frame display order, frame dependency sets, and queuing the dependency frames in advance so as to enable display of a video. The module enables a video to be played forwards or backwards at a variety of playback speeds from any position within the video. In one implementation, a device communicatively coupled to a plurality of decoders accesses a video that includes a plurality of frames. One or more of the frames are decodable by one or more of the communicatively coupled decoders. The device identifies a frame in the video that is to be displayed, and determines a plurality of dependency frames in the video upon which decoding of the frame to be displayed depends. The device provides an indication that one or more associated dependency frames are to be decoded.

Abstract: This is directed to systems, methods and computer-readable media for media files having timed and untimed location metadata. For example, a media file can include timed location metadata stored in a metadata track of the media file, such that individual timed packets of location information are each associated with different portions of the recorded media. In some embodiments, the location metadata can include information describing the direction the device is facing and/or elevation/tilt of the device relative a horizontal plane (e.g. a plane perpendicular to a line between the device position and the center of the earth) and/or a motion of the device.

Abstract: Disclosed herein are systems, methods, and non-transitory computer-readable storage media for creating and using media asset proxies. The media asset proxies represent a digital media asset and are created by filtering and modifying elements from the digital media asset. The media asset proxies can be queried in the same manner as their corresponding digital media asset.

Abstract: A video decoder includes a mode of operation for reverse playback that includes dividing a sequence of coded video data into groups of pictures (GOPs) and further subdividing the groups of pictures into a plurality of spans. Each span is processed in reverse coding order by decoding the first key frame and any reference frames that precede the currently processed span in coding order, selecting a plurality of frames from the currently processed span for decode and display and further selecting for decode but not display any reference frames from the currently processed span that were omitted by the first selection. The method accommodates control parameters that define the number of spans selected from each GOP and the number of frames selected from each span for decoded and display.

Abstract: A method is provided through which information about media files embedded in web pages is obtained from a server and analyzed to determine if the media files are playable on a device. This information may be external to the media file or may be a portion of the media file itself. The determination of whether a media file is playable by a device is made based on information obtained from the server without downloading the entirety of the media file.

Abstract: Techniques for synchronizing audio and video content for presentation to a user at a same rate are provided. Streams of content from two or more sources of media, each media source having an associated clock, are synchronized by a synchronizing component and processor with respect to a master clock. As well, techniques are provided for ensuring that output devices are synchronized at preview startup. That is, such techniques ensure that the output devices start playing the media at the same time as well as at the same rate.

Abstract: Methods, systems and machine readable storage medium for allowing playback of streaming media at playback rates of other than 1× are described. In one embodiment, a method can determine different sets of I-frames that are available before a display deadline, where each set can have a different cadence, and one of the sets can be selected for download and display to achieve playback at other than 1×. Byte range requests from a client device can be used to retrieve each of the I-frames. Other methods, system and media are also described.

Abstract: Video data with high frame rates may be displayed on devices with limited resources (e.g., decoder and/or display resources). These devices may have their resources devoted to other tasks or may not be capable to display the video data at the high frame rates. The coding method may include coding the frames such that additional droppable frames are included in the encoded video data. The decoding method may include dropping droppable frames before the encoded video data is decoded to reduce the number of frames that will be decoded and displayed. These methods may be applied to video data that has a variable frame rate and may be combined with processing the image sequence for slow motion playback.

Abstract: Methods, systems and machine readable storage medium for allowing playback of streaming media at playback rates of other than 1× are described. In one embodiment, a method can determine different sets of I-frames that are available before a display deadline, where each set can have a different cadence, and one of the sets can be selected for download and display to achieve playback at other than 1×. Byte range requests from a client device can be used to retrieve each of the I-frames. Other methods, system and media are also described.