Abstract: Objects within an image are assigned a semantic interest that is indicative of their importance to the image as a whole. The objects are detected automatically, and sub-images that span each of the objects are excerpted from the image. An image embedding is determined for each of the sub-images as well as for the whole image by using an image encoder implemented as a trained multi-modal neural network. The degree of similarity between the image embeddings of a sub-image and that of the whole image is used as a measure of the semantic importance of the object portrayed in the sub-image. Objects of high semantic importance comprise the semantic regions of interest of the image. Knowledge of such regions may be used to enhance and improve the efficiency of downstream image-processing tasks, such as image compression, pan and scan, and contrast enhancement.

Abstract: An operator of a digital audio workstation (DAW) application is able to assign individual tracks of a DAW session for export to specific players within a musical score of a scorewriter application. The DAW operator associates each track with a player identifier, which is retained in association with an interoperable format file generated by the export process. When the scorewriter imports such a file, it extracts the player identifier and uses it to map the track to a scored instrument. The mapping may also depend on a scorewriter arrangement of players for the instruments. The DAW operator may assign multiple tracks representing a given instrument played with different techniques to a single instrument part in a score. The playing techniques for the instruments are also associated with the tracks and may be parsed by the scorewriter to annotate the score with the corresponding notations.

Abstract: A media composition application, such as a musical scorewriter or a digital audio workstation, provides in situ suggestions for continuation or completion of a media composition. The suggestions are based on some or all of the portion of the composition already composed or are based on a corpus of compositions, such as those by a particular composer or those of a specific genre. The length of the suggestions is specified by the user. The suggestions are provided within a graphical user interface of the application and displayed as a possible direct continuation of the composition within a musical stave. If the user rejects the suggestion, additional suggestions are automatically displayed in situ. Reductive, most-probable suggestions may be offered as well as exploratory suggestions that facilitate a creative compositional interaction between user and application. Data filters enable selected aspects of a data source to be used for suggestion generation.

Abstract: A graphical user interface for music creation applications, such as score notation applications and digital audio workstations, includes multi-dimensional gestures. To enter a sound event into a musical project, a user uses an input device to select and drag a desired sound event in one or more dimensions. The relative position or rate of movement along a given dimension defines a value of a sound event parameter allocated to the given dimension. The sound event is entered into the project when the selection is released. The user inputs the gesture using a pointing device such as a mouse, stylus with a touch screen, or finger on a touch screen. Stylus dimensions mapped to sound event parameters may include, horizontal and vertical stylus tip positions, vertical and horizontal tilt of the stylus, and stylus tip pressure. Sound event parameters controlled by the gestures may include diatonic pitch, chromatic inflection, and duration.

Abstract: Automatic editing of media compositions is performed using media editing applications equipped with neural-network-based deep-learning models. The automatic editing is adapted to the practices of local users of a media editing application by training the models on a combination of media compositions previously edited by third-party media editors and media compositions edited by local users. Training data input vectors for the model comprise representative portions of a composition's raw media, and corresponding output vectors include values of parameters that define editing functions applied to the raw media to generate an edited media composition. A user interface enabling a user to adjust and monitor machine learning parameters is provided. Adaptive automatic editing may assist in the creation of video and audio compositions, as well as in the generation of musical scores.

Abstract: An operator of a digital audio workstation (DAW) application is able to assign individual tracks of a DAW session for export to specific players within a musical score of a scorewriter application. The DAW operator associates each track with a player identifier, which is retained in association with an interoperable format file generated by the export process. When the scorewriter imports such a file, it extracts the player identifier and uses it to map the track to a scored instrument. The mapping may also depend on a scorewriter arrangement of players for the instruments. The DAW operator may assign multiple tracks representing a given instrument played with different techniques to a single instrument part in a score. The playing techniques for the instruments are also associated with the tracks and may be parsed by the scorewriter to annotate the score with the corresponding notations.

Abstract: A graphical user interface of control software hosted by an audio interface device enables an operator to match a particular monitor mix with the input signals that contribute to the mix. The audio interface device receives live and non-live inputs and generates a plurality of monitor mixes. Using the graphical user interface, the operator selects a monitor mix to which a distinguishing graphical feature is assigned. In response, a graphical representation of faders representing the live and non-live inputs to the selected monitor mix automatically redisplays the faders with the distinguishing graphical feature and at an attenuation level corresponding to each input's contribution to the selected output mix. The distinguishing feature may be a color or a shading. The user interface may be used in the context of a studio recording session as well as in live performance settings.

Abstract: Low bitrate encoding of HDR-coded content with codecs that achieve compression by lowering the precision of encoding can result in significant loss of color fidelity. The problem stems from the logarithmic nature of the HDR encoding and consequent excessive quantization of color components of HDR-coded material. The loss is ameliorated by applying a saturation emphasis function that boosts color values of the HDR-coded material prior to compression. In various implementations the saturation emphasis function is a monotonically increasing function that preferentially boosts lower color values. Following compression and subsequent decompression, the color boost is reversed. In video post-production workflows proxies may be compressed to very low bitrates. For such workflows, the described methods significantly reduce codec-induced color artifacts in the decompressed proxy, thus restoring the ability of editors to make valid technical decisions based on the proxy during post-production.

Abstract: Indexing of media sources available to media composition applications, such as video editing applications and digital audio workstations, is extended to sources of an arbitrary type, including non-file type sources. Examples of such sources include devices using baseband video and audio protocols, network ports and physical devices connected via USB, Thunderbolt, etc. The sources are discovered and indexed even when not previously known to the application with the result that any available bitstream becomes available to the application. Applications access the sources via feeders, which are addressable portals through which media essence is received. The indexing methods involve the instantiation of a hierarchy of plug-in software modules by a format-handling subsystem of the application or by a media indexing service servicing one or more applications.

Abstract: A universal media difference engine generates a change list specifying the edits required to create an edited revision of a media composition from a base version. The difference engine determines the format of the media composition, locates and installs a plug-in corresponding to the format, and uses the plug-in to parse the composition and generate the change list. The supported compositional formats include formats native to specific media editing applications, as well as interoperable formats. The difference engine is able to convert rich change lists expressed in native form to canonical change lists that are compatible with multiple editing applications. Timeline, mixer configuration, and scene graph composition types are supported. Content management system storage requirements are reduced by storing a base version and change lists instead of multiple revisions of the composition. A media composition recreation engine recreates an edited revision by applying a change list to a prior version.

Abstract: An audio signal processing graph is automatically recomposed to optimize its properties by reassigning and instantiating its nodes on available locales such that the number of inter-locale connections is minimized and latency between inputs, such a performer's live input, and outputs, such as a monitored recording mix, is minimized. The recomposition exploits associative, commutative, and decomposition properties of certain node types, including mixer nodes. A graph recomposition may decompose a mixer node into a first stage instantiated by a software plug-in hosted by a computer running a digital audio workstation in a first locale, and a second stage assigned to an audio processing device in a second locale. Automatic signal graph recomposition occurs when the system is initialized, the graph is reconfigured, a new desired behavior of the graph is specified, or the available network resources are changed.

Abstract: Indexing of media sources available to media composition applications, such as video editing applications and digital audio workstations, is extended to sources of an arbitrary type, including non-file type sources. Examples of such sources include devices using baseband video and audio protocols, network ports and physical devices connected via USB, Thunderbolt, etc. The sources are discovered and indexed even when not previously known to the application with the result that any available bitstream becomes available to the application. Applications access the sources via feeders, which are addressable portals through which media essence is received. The indexing methods involve the instantiation of a hierarchy of plug-in software modules by a format-handling subsystem of the application or by a media indexing service servicing one or more applications.

Abstract: A graphical user interface for music creation applications, such as score notation applications and digital audio workstations, includes multi-dimensional gestures. To enter a sound event into a musical project, a user uses an input device to select and drag a desired sound event in one or more dimensions. The relative position or rate of movement along a given dimension defines a value of a sound event parameter allocated to the given dimension. The sound event is entered into the project when the selection is released. The user inputs the gesture using a pointing device such as a mouse, stylus with a touch screen, or finger on a touch screen. Stylus dimensions mapped to sound event parameters may include, horizontal and vertical stylus tip positions, vertical and horizontal tilt of the stylus, and stylus tip pressure. Sound event parameters controlled by the gestures may include diatonic pitch, chromatic inflection, and duration.

Abstract: Near real-time collaborative media production by parties located remotely from each other is facilitated by the described methods. Automated dialog replacement with voice talent, editing system operator, and director located remotely from each other is supported by generating synchronized near real-time feeds for the operator and director. Pre-recorded media played back from the editing system is streamed without delay to the talent. A pre-recorded media feed is also delayed to synchronize it with the incoming talent stream, which was recorded in sync with the pre-recorded media stream when received by the talent. The synchronized feed is output to the operator and streamed to the director. Talkback channels and webcam video with appropriate synchronization delays support communication among the parties.

Abstract: Indexing of media sources available to media composition applications, such as video editing applications and digital audio workstations, is extended to sources of an arbitrary type, including non-file type sources. Examples of such sources include devices using baseband video and audio protocols, network ports and physical devices connected via USB, Thunderbolt, etc. The sources are discovered and indexed even when not previously known to the application with the result that any available bitstream becomes available to the application. Applications access the sources via feeders, which are addressable portals through which media essence is received. The indexing methods involve the instantiation of a hierarchy of plug-in software modules by a format-handling subsystem of the application or by a media indexing service servicing one or more applications.

Abstract: Automatic editing of media compositions is performed using media editing applications equipped with neural-network-based deep-learning models. The automatic editing is adapted to the practices of local users of a media editing application by training the models on a combination of media compositions previously edited by third-party media editors and media compositions edited by local users. Training data input vectors for the model comprise representative portions of a composition's raw media, and corresponding output vectors include values of parameters that define editing functions applied to the raw media to generate an edited media composition. A user interface enabling a user to adjust and monitor machine learning parameters is provided. Adaptive automatic editing may assist in the creation of video and audio compositions, as well as in the generation of musical scores.

Abstract: Efficient processing of chroma-subsampled video is performed using convolutional neural networks (CNNs) in which the luma and chroma channels are processed separately. The luma channel is independently convolved and downsampled and, in parallel, the chroma channels are convolved and then merged with the downsampled luma to generate encoded chroma-subsampled video. Further processing of the encoded video that involves deconvolution and upsampling, splitting into two sets of channels, and further deconvolutions and upsampling is used in CNNs to generate decoded chroma-subsampled video in compression-decompression applications, to remove noise from chroma-subsampled video, or to upsample chroma-subsampled video to RGB 444 video. CNNs with separate luma and chroma processing in which the further processing includes additional convolutions and downsampling may be used for object recognition and semantic search in chroma-subsampled video.

Abstract: A media composition application, such as a musical scorewriter or a digital audio workstation, provides in situ suggestions for continuation or completion of a media composition. The suggestions are based on some or all of the portion of the composition already composed or are based on a corpus of compositions, such as those by a particular composer or those of a specific genre. The length of the suggestions is specified by the user. The suggestions are provided within a graphical user interface of the application and displayed as a possible direct continuation of the composition within a musical stave. If the user rejects the suggestion, additional suggestions are automatically displayed in situ. Reductive, most-probable suggestions may be offered as well as exploratory suggestions that facilitate a creative compositional interaction between user and application. Data filters enable selected aspects of a data source to be used for suggestion generation.

Abstract: Automatic editing of media compositions is performed using media editing applications equipped with neural-network-based deep-learning models. The automatic editing is adapted to the practices of local users of a media editing application by training the models on a combination of media compositions previously edited by third-party media editors and media compositions edited by local users. Training data input vectors for the model comprise representative portions of a composition's raw media, and corresponding output vectors include values of parameters that define editing functions applied to the raw media to generate an edited media composition. A user interface enabling a user to adjust and monitor machine learning parameters is provided. Adaptive automatic editing may assist in the creation of video and audio compositions, as well as in the generation of musical scores.

Abstract: An overlay surface for a user interface of a device comprises a lower light-diffusing semitransparent layer affixed to an upper transparent layer. Graphics printed on the bottom surface of the transparent layer are protected from wear by the overlying transparent layer, while light sources back-illuminating the graphics from below are diffused by the semitransparent layer before they reach the graphics. The graphics of the surface may be provided by an opaque mask having transparent or translucent cutouts. The multilayer surface enables the graphics to be evenly back-illuminated while at the same time being protected from external wear without the need for additional light-diffusing elements such as light pipes.