Abstract: A user-configurable modular audio control surface comprises master modules for controlling global surface properties and channel modules for controlling one or more individual audio channels. The modules are disposed in a two-dimensional spatial arrangement such that any module can occupy a location within the control surface not occupied by another module. The modules are connected to each other and to external platforms hosting media applications and plug-ins via a network. Control surface users can interact with external applications via remote graphical user interfaces displayed on modules within the surface, and can automate multiple external applications using an automation system built into the surface. Automation line graphs and metadata for both internal and external applications are displayed over the corresponding waveform displays that can include audio ahead of a current playback location.

Abstract: In current systems, augmented reality graphics is generated at a central broadcast facility or studio where it is combined with the video that is transmitted to subscribers. By contrast, in the described system, the studio does not generate the graphics, but transmits video together with real-time metadata to the end-user set-top device. The end-user device generates the augmented reality graphics, using the metadata to determine positional and other parameters for displaying the graphics. Shifting the generation of augmented reality graphics to the consumer level facilitates end-user customization and individualized targeting of information by a broadcaster or advertiser.

Abstract: In current systems, augmented reality graphics is generated at a central broadcast facility or studio where it is combined with the video that is transmitted to subscribers. By contrast, in the described system, the studio does not generate the graphics, but transmits video together with real-time metadata to the end-user set-top device. The end-user device generates the augmented reality graphics, using the metadata to determine positional and other parameters for displaying the graphics. Shifting the generation of augmented reality graphics to the consumer level facilitates end-user customization and individualized targeting of information by a broadcaster or advertiser.

Abstract: Methods of converting an open group of pictures (GOP) to a Closed GOP are described that remove backward references to the GOP prior to the GOP undergoing conversion in an Open GOP interframe-encoded video stream. Within the GOP undergoing conversion, bipredictive frames with backward references to the prior GOP are replaced with frames referring only to the I frame of the GOP undergoing conversion, which is flagged as an IDR frame. Consolidation and assembly of video clips that are encoded using Open GOP compression schemes generally result in broken interframe references between GOPs. To address this, existing methods involve significant decoding and re-encoding of edited video sequences. The described methods serve to break the backward chain of references in the encoded stream, resulting in reduced re-encoding requirements, with attendant reduced quality degradation of the edited sequence.

Abstract: A user-configurable modular audio control surface comprises master modules for controlling global surface properties and channel modules for controlling one or more individual audio channels. The modules are disposed in a two-dimensional spatial arrangement such that any module can occupy a location within the control surface not occupied by another module. The modules are connected to each other and to external platforms hosting media applications and plug-ins via a network. Control surface users can interact with external applications via remote graphical user interfaces displayed on modules within the surface, and can automate multiple external applications using an automation system built into the surface. Automation line graphs and metadata for both internal and external applications are displayed over the corresponding waveform displays that can include audio ahead of a current playback location.

Abstract: A collaboration server hosts software for collaborative composition and editing of a media project with project collaborators using different media editing applications each having their own native data format. Project collaborators, such as video editors, sound editors, effects and graphics artists, and producers access a shared project workspace which contains a snapshot of the current state of the media project in a canonical format, as well as source media files, native application metadata, and change notes. Each editing application includes a module enabling it to read the canonical snapshot representation, and also to flatten its native data model representation into the canonical representation for writing to the shared project workspace. A collaboration server hosts the shared project space, and includes a workflow manager for issuing change notifications and handling versions, and an application server for the shared project user interface.

Abstract: Systems and methods for linking time-based media and temporal metadata provide single command control during editing of media having associated temporal metadata. A single control enables an editor to switch between monitoring both source media and its corresponding temporal metadata and monitoring both a pre-recorded version of the media and its corresponding temporal metadata. Another single control enables an editor to start and stop the recording of a source media track and its corresponding source temporal metadata. In one application, the editing is performed for media tracks having time-based spatial metadata for playback in immersive environments with the spatial metadata defining an apparent location of sound objects within the immersive environment.

Abstract: Synchronous playback of time-based media received from one or more locations remote from a primary editing/mixing studio is achieved by time-stamping media samples with a local presentation time before streaming them to the primary studio. At the primary studio, samples having the same presentation timestamp are played back at the same time, independently of the samples' arrival time at the playback system. Media stored locally to the playback system may also be included as part of the synchronous playback using locally computed presentation times. In order to accommodate media streaming transmission delays, the playback system negotiates a suitable delay with the remote systems such that samples corresponding to a given presentation time are received at the playback system from remote locations prior to playback of media corresponding to the given presentation time.

Abstract: A synchronized media project, synchronization SDK, and synchronization service enable multiple users to produce a media composition collaboratively. A media project includes composition assets that reference media assets. A synchronization SDK enables multiple applications to edit the media project independent of an internal organization of the project. A project synchronization service synchronizes copies of media projects located on clients with a server-based versioned master project. User-selectable project synchronization modes determine whether media project assets are stored on clients, on the server, or on both, and when media assets are transferred. Synchronized media projects include composition assets in proprietary formats and in interoperable formats. Interoperable composition assets are rendered from proprietary composition assets on a track-by-track basis.

Abstract: Methods and systems for populating a cache memory that services a media composition system. Caching priorities are based on a state of the media composition system, such as media currently within a media composition timeline, a composition playback location, media playback history, and temporal location within clips that are included in the composition. Caching may also be informed by descriptive metadata and media search results within a media composition client or a within a media asset management system accessed by the client. Additional caching priorities may be based on a project workflow phase or a client project schedule. Media may be partially written to or read from cache in order to meet media request deadlines. Caches may be local to a media composition system or remote, and may be fixed or portable.

Abstract: Large numbers of similar-looking or identical physical controls on an audio control surface can make it difficult for operators to rapidly locate desired controls. Highlighting of selected controls focuses the operator's visual attention on controls the operator is manipulating, or has just manipulated and is likely to manipulate again. Controls peripheral to or otherwise related to manipulated controls may also be highlighted. Highlighting methods include the use of control illumination intensity, color, and labeling display text. Visual focusing may include highlighting relationships between channel strips or between individual physical controls, or between the physical controls of a surface and one or more digital audio workstations.

Abstract: Video editing methods and systems enable an editor to edit a video project for which source media assets are located at a media storage server located remotely from the editor with substantially the same fidelity and editing feature set that would be available if the source media assets and editor were co-located. A video editing client used by the editor maintains a persistent cache of proxy media with the layers of the video project stored independently, facilitating editing with combinations locally originated assets and remote assets. The client requests frames not already cached from the remote server via a low bandwidth network. Unless a frame is purged from the cache, no frame is requested from the server more than once. A multi-level priority prefetching scheme, including sequence-based prefetching, populates the cache with frames likely to be requested during editing.

Abstract: A distributed self-scaling network audio processing system includes end nodes interconnected by packet-switched network and operating as peers on the network. Each of the end nodes supports local input processing, mixing, and output processing. The input processing includes the option of dual input channels for supporting separate front-of-house and monitor workflows. End nodes are added to the system to support specific audio processing applications, based on the number of audio sources, the number of output mixes required, and the number of locations from which users choose to interact with the system.

Abstract: During the production of a time-based media project, it is often desirable for editors to work with media files or reels of a given size, both in terms of the temporal duration of media represented in each file and the number of tracks in a file. During the course of editing, files may become longer, or incorporate additional tracks, making them cumbersome to handle. A super-file view that displays multiple files simultaneously provides a framework for an editor to rebalance files during the course of media production. A graphical user interface permits users to adjust the content of the various files, including moving tracks among multiple files that comprise a given reel, as well as media between files belonging to different reels.

Abstract: A synchronized media project, synchronization SDK, and synchronization service enable multiple users to produce a media composition collaboratively. A media project includes composition assets that reference media assets. A synchronization SDK enables multiple applications to edit the media project independent of an internal organization of the project. A project synchronization service synchronizes copies of media projects located on clients with a server-based versioned master project. User-selectable project synchronization modes determine whether media project assets are stored on clients, on the server, or on both, and when media assets are transferred. Synchronized media projects include composition assets in proprietary formats and in interoperable formats. Interoperable composition assets are rendered from proprietary composition assets on a track-by-track basis.

Abstract: A computer-based method for media composition of a family of related time-based media programs. The method involves creating a master program with time-based elements of video and/or audio as well as time-based and non-time-based metadata, creating a derivative program that includes derivative elements, defining an inheritance relationship between the master program and the derivative program that specifies elements of the master program to be inherited by the derivative program, and causing the derivative program to inherit the specified elements from the master program in accordance with the inheritance relationship. User interfaces are provided for creating, editing, and viewing hierarchical trees of related programs.

Abstract: Multiple applications request data from multiple storage units over a computer network. The data is divided into segments and each segment is distributed randomly on one of several storage units, independent of the storage units on which other segments of the media data are stored. At least one additional copy of each segment also is distributed randomly over the storage units, such that each segment is stored on at least two storage units. This random distribution of multiple copies of segments of data improves both scalability and reliability. When an application requests a selected segment of data, the request is processed by the storage unit with the shortest queue of requests. Random fluctuations in the load applied by multiple applications on multiple storage units are balanced nearly equally over all of the storage units.

Abstract: A system for transferring metadata between an audio mixing console and a digital audio workstation connected by an audio link and a computer network. The audio mixing console and the digital audio system implement an Ethernet-based communication protocol for sending messages between them over the network connection, the messages including metadata specifying a plurality of settings on the audio mixing console corresponding to audio information received by the audio mixing console during a live performance. The audio information is sent to the digital audio system over the high bandwidth connection. The high bandwidth connection can also be used to transfer the metadata. The transfer of metadata between the console and the digital audio workstation facilitates new digital audio archiving, virtual live mixing, and virtual soundcheck workflows.

Abstract: Methods and systems for constructing media output by combining multiple input media channels corresponding to different views of a scene. The construction involves combining channels that have different spatial ranges and/or color ranges so as to produce a constructed output having a range that is a composite of input channel spatial and/or color ranges. The constructed output is generated dynamically from available input channels in response to output requirements specifying a view defined in part by spatial and/or color parameter ranges. Available input channels are represented by a single multi-channel source object comprising a hierarchy of grouped and aligned channels, with individual media essence files at the lowest level. Input channel ranges and relationships among the ranges are specified by metadata associated with each channel. During editing, input channels may be added, removed, or changed and the range and nature of the constructed media output are updated dynamically.

Abstract: High dynamic range media sensor output is encoded using a hybrid transfer function. The hybrid transfer function conforms to existing industry standards over standard brightness level ranges, such as between black and white reference points, and uses a different function to encode captured signals having intensities below the black point and above the white point. Precision over the expanded dynamic range of captured image data is optimally preserved using a two-byte per pixel representation. The transfer function varies across the full dynamic range without clamping off below the maximum value of the expanded captured intensity range. One hybrid transfer function conforms to BT.1886 up to the white point, and uses a linearly varying encoding gamma above the white point up to 1300% IRE, producing encoded values in a 2.14 fixed point representation. The hybrid transfer function is continuous at the white point, and may also have a continuous gradient there.