Abstract: When a portion of an original audio track is unsatisfactory, alternate portions are searched for using phonetic matching within phonetically indexed alternate files. The alternates may correspond to recordings captured in a different take from the original where timecode matching is unavailable. An editor may preview one or more candidate alternates and select a preferred one to be used to replace the original. A media editing application, such as a digital audio workstation, automatically aligns and matches the preferred alternative to the original using waveform matching and optional gain matching.

Abstract: An audio signal processing graph is automatically recomposed to optimize its properties by reassigning and instantiating its nodes on available locales such that the number of inter-locale connections is minimized and latency between inputs, such a performer's live input, and outputs, such as a monitored recording mix, is minimized. The recomposition exploits associative, commutative, and decomposition properties of certain node types, including mixer nodes. A graph recomposition may decompose a mixer node into a first stage instantiated by a software plug-in hosted by a computer running a digital audio workstation in a first locale, and a second stage assigned to an audio processing device in a second locale. Automatic signal graph recomposition occurs when the system is initialized, the graph is reconfigured, a new desired behavior of the graph is specified, or the available network resources are changed.

Abstract: Tagged sound file libraries are searched using an iterative search application. A searcher specifies a multi-term search query which is directed to the sound libraries. Responsive audio files from the libraries are ranked by confidence levels for the occurrence of sounds corresponding to searched terms, weighted by user-determined search term priorities. High-ranking candidate audio files are presented to the user who is able to indicate a preference or a disfavoring of the candidates. Based on the feedback, the application adjusts the priorities of the individual search terms in the search query, adds or removes search terms from the query, and performs a new search with a revised set of search terms. The results are ranked using the adjusted priorities. The iterative search locates files with desired sounds that conform to the initial search or that have evolved from it under the user's direction.

Abstract: An audio signal processing graph is automatically recomposed to optimize its properties by reassigning and instantiating its nodes on available locales such that the number of inter-locale connections is minimized and latency between inputs, such a performer's live input, and outputs, such as a monitored recording mix, is minimized. The recomposition exploits associative, commutative, and decomposition properties of certain node types, including mixer nodes. A graph recomposition may decompose a mixer node into a first stage instantiated by a software plug-in hosted by a computer running a digital audio workstation in a first locale, and a second stage assigned to an audio processing device in a second locale. Automatic signal graph recomposition occurs when the system is initialized, the graph is reconfigured, a new desired behavior of the graph is specified, or the available network resources are changed.

Abstract: Playback of a graphical representation of a digital musical score is synchronized with an expressive audio rendering of the score that contains tempo and dynamics beyond those specified in the score. The method involves determining a set of offsets for occurrences of score events in the audio rendering by comparing and temporally aligning audio waveforms of successive subclips of the audio rendering with corresponding audio waveforms of successive subclips of an audio rendering synthesized directly from the score. Tempos and dynamics of human performances may be extracted and used to generate expressive renderings synthesized from the corresponding digital score. This enables parties who wish to distribute or share music scores, such as composers and publishers, to allow prospective licensors to evaluate the score by listening to an expressive musical recording instead of a mechanically synthesized rendering.

Abstract: Temporal regions of a time-based media program that contain spoken dialog in a language that is dubbed from a primary language are identified automatically. A primary language audio track of the media program is compared with an alternate language audio track. Closely similar regions are assumed not to contain dubbed dialog, while the temporal inverse of the similar regions are candidate regions for containing dubbed speech. The candidate regions are provided to a dub validator to facilitate locating each region to be validated without having to play back or search the entire time-based media program. Corresponding regions of the primary and alternate language tracks that are closely similar and that contain voice activity are candidate regions of forced narrative, and the temporal locations of these regions may be used by a validator to facilitate rapid validation of forced narrative in the program.

Abstract: Information about each of the media assets that contribute to a published media composition is needed to determine attribution and contractual terms for the composition as a whole. The generation of a hierarchical asset genealogy that includes information about each of the assets that contribute to a media composition provides a mechanism for recording the full asset contribution history of a composition. The information may include asset ownership, distribution rights, asset structure, talent, and editors involved in generating the asset. For time-based compositions, it may specify which portions of an asset contribute to the composition. A report generator generates asset reports from the asset genealogy without the need to access or read the original composition, which may be represented by a proprietary compositional data model corresponding to an editing applications used to create the composition.

Abstract: The described methods, systems, and computer program products enable media composition editors to tag compositional objects with editorial rendition rules that specify how rendering decisions are to be made and how source media assets are to be selected and processed to generate an output rendering of a media composition that adheres to the editor's rendition-related choices. The editorially-specified rendition rules include spatial-type rules, temporal-type rules, and color-type rules, and generally pertain to the spatial, temporal, and color ranges and resolutions to be included within a rendition. When a rendition of a media composition is to be generated, a rendition engine interprets and applies the editorial rendition rules to generate and output the rendition in accordance with rendition parameters supplied to the rendition engine by a rendition profile, and, in some cases, by the media composition and the source media assets.

Abstract: The described methods, systems, and computer program products enable media composition editors to tag compositional objects with editorial rendition rules that specify how rendering decisions are to be made and how source media assets are to be selected and processed to generate an output rendering of a media composition that adheres to the editor's rendition-related choices. The editorially-specified rendition rules include spatial-type rules, temporal-type rules, and color-type rules, and generally pertain to the spatial, temporal, and color ranges and resolutions to be included within a rendition. When a rendition of a media composition is to be generated, a rendition engine interprets and applies the editorial rendition rules to generate and output the rendition in accordance with rendition parameters supplied to the rendition engine by a rendition profile, and, in some cases, by the media composition and the source media assets.

Abstract: Within a media production environment, multiple variants of a given media asset may be stored on various systems within the environment. The described production environment tracks whether a newly created asset is a variant of an original asset within the environment, and, if so, assigns it a linkage key that it shares with the original and any other variants of the original. When responding to an asset search request, responsive assets may include assets sharing a linkage key. These are assembled into an aggregate asset that is returned as a single asset among the search results. Aggregate assets include descriptive metadata copied from its constituent variants and shown in the search results to represent the asset within the search results. Using links within the aggregate asset, the application automatically retrieves a compatible variant of the selected asset, if one is present within the production environment.

Abstract: A user interface for a digital audio workstation provides an overview of the audio signal routing of an audio composition in the form of a node graph. The node graph updates in real time as an audio session is edited. The representation of the nodes on the graph indicates the node type, such as audio input or track, mixer, plug-in, and output, as well as the processing resources assigned to each node. The node graph includes one or more nodes representing submixes that may be adjusted using a mixer channel independently of other submixes or outputs of the audio session. The representation of audio signal flow between the nodes in the graph distinguishes between insert routing and auxiliary sends. The user interface may be used interactively to edit the audio composition by providing a toolbox for creating new nodes and commands for specifying audio signal connections between nodes.

Abstract: A modular video server includes a Video Blade module and a separate physical layer module. The Video Blade performs video processing functions, such as video encoding and decoding, resizing, slow motion, and color space conversion, and the physical layer module implements the physical layer for input and output of video from devices external to the video server. The modular design permits the type of Video Blade and the type of physical layer module to be specified independently of each other, with each type of Video Blade being compatible with each type of physical layer module. This reduces the effort and cost of providing new video servers that support evolving video processing requirements and new video transmission standards.

Abstract: A collaboration server hosts software for collaborative composition and editing of a media project with project collaborators using different media editing applications each having their own native data format. Project collaborators, such as video editors, sound editors, effects and graphics artists, and producers access a shared project workspace which contains a snapshot of the current state of the media project in a canonical format, as well as source media files, native application metadata, and change notes. Each editing application includes a module enabling it to read the canonical snapshot representation, and also to flatten its native data model representation into the canonical representation for writing to the shared project workspace. A collaboration server hosts the shared project space, and includes a workflow manager for issuing change notifications and handling versions, and an application server for the shared project user interface.

Abstract: Plug-in software modules for a media processing application are deployed in a service-oriented architecture as microservices within a Web-scale environment. Each plug-in module registered within a distributed service hosting environment runs as an instance within a gasket that mediates the interaction of the plug-in instance with the environment. The gaskets convert media essence that is input to or output from the plug-in instances from media files to buffers and vice versa as required. Media essence representing intermediate results may be streamed as media essence channels on a data bus, enabling microservice-based processing to be applied in a live environment. Gaskets subscribe to specific media essence channels on the data bus, and output processed media to a media essence channel corresponding to the function of the plug-in module. The use of plug-in associated media channels on a data bus supports decoupled media processing by the plug-in modules.

Abstract: A media file indexing service involves maintaining a media file index in a location external to the media file itself. The index is built up incrementally as the media file is being captured or transferred to shared storage. Client systems receive raw messages with information as to the location within the media file of media elements within each media file portion after it is written to shared storage, or receive refined messages indicating changes to the media file without the associated location information. Clients may query the index maintained by the index service for specific media elements during file capture, enabling edit-while-capture workflows.

Abstract: A software architecture and framework based on plug-in software modules supports flexible handling of synchronous data streams by media production and editing applications. Plug-ins called by the applications convert data from synchronous data streams into a form that enables a user of such an application to view and edit time-synchronous data contained within such data streams. The synchronous data is displayed in a temporally aligned manner in a synchronous data track within a timeline display of the application user interface. In one example, closed caption data extracted from the ancillary portion of a video signal is displayed as text on a data track temporally synchronized with the source video track. Other plug-ins analyze media tracks to generate time-synchronous data which may also be displayed in a temporally aligned manner within a synchronous data track in a timeline.

Abstract: Augmented reality enables an operator to visualize values of parameters of audio channels during audio mixing. A heads-up display worn by the operator, such as smart glasses, displays virtual graphical objects representing the parameters so that they appear within a three-dimensional space surrounding the operator and an audio mixing console. Parameter values are represented by the location, size, and other attributes of the virtual graphical objects. The operator adjusts the parameter values using physical and touch controls on the console and by manipulating the virtual objects with a body part such as a finger. Sensors mounted on the heads-up display and on other body parts capture position and movement of the operator in real time and send the captured data to a system running augmented reality control software. Graphical user interfaces of a media processing application controlled by the console and of plug-in modules may also be displayed on the heads-up display.

Abstract: A unique material identifier (UMID) for a media file that was not provided with a UMID at its point of origination is generated by using the content of the file, and is independent of the time of file import or accessing. For a given item of media material, the UMID remains unchanged and uniquely identifies the item when such a file is imported or accessed multiple times. The UMID may be generated by hashing together selected portions of the metadata and essence of the media file. The amount of metadata and essence sampled is chosen to provide a high degree of assurance that the UMID will be unique, but is kept small enough so as to avoid causing a perceptible lag when the UMID is generated. In various embodiments the UMID is based purely on one or more selected portions of the media file essence.

Abstract: A synchronized media project, synchronization SDK, and synchronization service enable multiple users to produce a media composition collaboratively. A media project includes composition assets that reference media assets. A synchronization SDK enables multiple applications to edit the media project independent of an internal organization of the project. A project synchronization service synchronizes copies of media projects located on clients with a server-based versioned master project. User-selectable project synchronization modes determine whether media project assets are stored on clients, on the server, or on both, and when media assets are transferred. Synchronized media projects include composition assets in proprietary formats and in interoperable formats. Interoperable composition assets are rendered from proprietary composition assets on a track-by-track basis.

Abstract: A synchronized media project, synchronization SDK, and synchronization service enable multiple users to produce a media composition collaboratively. A media project includes composition assets that reference media assets. A synchronization SDK enables multiple applications to edit the media project independent of an internal organization of the project. A project synchronization service synchronizes copies of media projects located on clients with a server-based versioned master project. User-selectable project synchronization modes determine whether media project assets are stored on clients, on the server, or on both, and when media assets are transferred. Synchronized media projects include composition assets in proprietary formats and in interoperable formats. Interoperable composition assets are rendered from proprietary composition assets on a track-by-track basis.