Abstract: Objects within an image are assigned a semantic interest that is indicative of their importance to the image as a whole. The objects are detected automatically, and sub-images that span each of the objects are excerpted from the image. An image embedding is determined for each of the sub-images as well as for the whole image by using an image encoder implemented as a trained multi-modal neural network. The degree of similarity between the image embeddings of a sub-image and that of the whole image is used as a measure of the semantic importance of the object portrayed in the sub-image. Objects of high semantic importance comprise the semantic regions of interest of the image. Knowledge of such regions may be used to enhance and improve the efficiency of downstream image-processing tasks, such as image compression, pan and scan, and contrast enhancement.

Abstract: A universal media difference engine generates a change list specifying the edits required to create an edited revision of a media composition from a base version. The difference engine determines the format of the media composition, locates and installs a plug-in corresponding to the format, and uses the plug-in to parse the composition and generate the change list. The supported compositional formats include formats native to specific media editing applications, as well as interoperable formats. The difference engine is able to convert rich change lists expressed in native form to canonical change lists that are compatible with multiple editing applications. Timeline, mixer configuration, and scene graph composition types are supported. Content management system storage requirements are reduced by storing a base version and change lists instead of multiple revisions of the composition. A media composition recreation engine recreates an edited revision by applying a change list to a prior version.

Abstract: A large language model (LLM) is used to broaden a search for supporting material for a media project. The LLM is provided with contextual material from the media project and optional grounding material and generates a list of types of supporting material. The list is provided to a machine-learning-based encoder, which encodes the list items into embedding space vectors. A body of source material, which may multimodal, is also encoded into the embedding space. A search engine is used to locate items of source material having embedded space vectors closest to vectors corresponding to the list of types of supporting material. Located items are provided to a media editing application as suggested supporting material. The body of source material may include live external sources. A small language model fine-tuned with a training data set generated by a LLM may be used instead of the LLM.

Abstract: Indexing of media sources available to media composition applications, such as video editing applications and digital audio workstations, is extended to sources of an arbitrary type, including non-file type sources. Examples of such sources include devices using baseband video and audio protocols, network ports and physical devices connected via USB, Thunderbolt, etc. The sources are discovered and indexed even when not previously known to the application with the result that any available bitstream becomes available to the application. Applications access the sources via feeders, which are addressable portals through which media essence is received. The indexing methods involve the instantiation of a hierarchy of plug-in software modules by a format-handling subsystem of the application or by a media indexing service servicing one or more applications.

Abstract: A media editing application is able to populate bins with media project assets that are responsive to a user query. The query is directed to a knowledge graph that includes nodes representing a diverse range of entity types associated with the media project and relationships between the nodes. The results returned by the knowledge graph are used to specify assets that are imported into the media editing application and placed in media project bins. Bins containing assets responsive to a user query extend an editor's reach to production elements in data sources beyond those represented in prior editorial asset systems and support ad-hoc queries that are not anticipated during when media project files are initially configured.

Abstract: A four-axis joystick provides the ability to control four parameters in a single input device. In one embodiment, the Z-direction of control is implemented by a telescoping joystick handle, with the X- and Y-directions of control being implemented by pivoting motions of the handle in directions radial to an axis of the joystick handle, and the theta-direction of control implemented by rotation of a portion of the handle about the handle axis. The four-axis joystick may be used as an input device for audio mixing applications with the X-, Y-, and Z-dimensions of control mapped to panning of an audio source in the left-right, forward-backward, and up-down directions respectively, and the theta-dimension mapped to volume control of low frequency effects.

Abstract: Dynamic user-defined stave filters enable users of scorewriter software applications to generate de-condensed musical parts from condensed scores. When a stave filter is applied to some or all of a stave displaying multiple types of musical element, one or more of the musical element types are hidden from view. Stave filters remove from display note and/or non-note element types. Filtered staves are linked to corresponding unfiltered staves such that compositional changes in a condensed score are reflected in the linked de-condensed staves. Uses include generation of individual parts from a score or generation of different orchestrations or arrangements. Filtered staves are drawn in an intelligent fashion to ensure that musical data are displayed in a musically valid form that conforms to stylistic conventions.

Abstract: A four-axis joystick provides the ability to control four parameters in a single input device. In one embodiment, the Z-direction of control is implemented by a telescoping joystick handle, with the X- and Y-directions of control being implemented by pivoting motions of the handle in directions radial to an axis of the joystick handle, and the theta-direction of control implemented by rotation of a portion of the handle about the handle axis. The four-axis joystick may be used as an input device for audio mixing applications with the X-, Y-, and Z-dimensions of control mapped to panning of an audio source in the left-right, forward-backward, and up-down directions respectively, and the theta-dimension mapped to volume control of low frequency effects.

Abstract: A software plug-in module that interfaces to a media editing host application generates and embeds information about a media composition being edited directly within portions of the user interface generated by the host application. The information may include a custom representation of media data of a time-based element of the media composition that replaces, augments, or overlays a timeline representation of the element generated by the host application. Media editing functionality provided by the plug-in may be accessed by an operator based on viewing or interacting with the custom representation. Results of analysis of the media composition by the plug-in may be displayed within the host-generated timeline and used by an operator as a basis for performing edit operations with standard host tools or with plug-in generated tools. Plug-ins may embed their interfaces within user interfaces of host digital audio workstations, non-linear video editing systems, and music notation applications.

Abstract: An operator of a digital audio workstation (DAW) application is able to assign individual tracks of a DAW session for export to specific players within a musical score of a scorewriter application. The DAW operator associates each track with a player identifier, which is retained in association with an interoperable format file generated by the export process. When the scorewriter imports such a file, it extracts the player identifier and uses it to map the track to a scored instrument. The mapping may also depend on a scorewriter arrangement of players for the instruments. The DAW operator may assign multiple tracks representing a given instrument played with different techniques to a single instrument part in a score. The playing techniques for the instruments are also associated with the tracks and may be parsed by the scorewriter to annotate the score with the corresponding notations.

Abstract: Music creation applications enable users to enter new objects into a musical project using gestures mediated by input maps that are adapted to the musical context of the object entry location. The musical context may be defined by an explicit aspect of the musical project or implicitly by notes or intervals already entered into the project. The adaptation of the input maps may include spatial scaling to provide greater space to objects favored by the context. The context-adapted input maps may be displayed and used to guide user input, such as with gestures on a touch screen. In some implementations, input-sensitive regions of input maps are scaled in accordance with the musical context without being displayed. The methods apply to musical object entry via pop-up input maps and virtual instrument displays.

Abstract: Objects within an image are assigned a semantic interest that is indicative of their importance to the image as a whole. The objects are detected automatically, and sub-images that span each of the objects are excerpted from the image. An image embedding is determined for each of the sub-images as well as for the whole image by using an image encoder implemented as a trained multi-modal neural network. The degree of similarity between the image embeddings of a sub-image and that of the whole image is used as a measure of the semantic importance of the object portrayed in the sub-image. Objects of high semantic importance comprise the semantic regions of interest of the image. Knowledge of such regions may be used to enhance and improve the efficiency of downstream image-processing tasks, such as image compression, pan and scan, and contrast enhancement.

Abstract: An operator of a digital audio workstation (DAW) application is able to assign individual tracks of a DAW session for export to specific players within a musical score of a scorewriter application. The DAW operator associates each track with a player identifier, which is retained in association with an interoperable format file generated by the export process. When the scorewriter imports such a file, it extracts the player identifier and uses it to map the track to a scored instrument. The mapping may also depend on a scorewriter arrangement of players for the instruments. The DAW operator may assign multiple tracks representing a given instrument played with different techniques to a single instrument part in a score. The playing techniques for the instruments are also associated with the tracks and may be parsed by the scorewriter to annotate the score with the corresponding notations.

Abstract: A media composition application, such as a musical scorewriter or a digital audio workstation, provides in situ suggestions for continuation or completion of a media composition. The suggestions are based on some or all of the portion of the composition already composed or are based on a corpus of compositions, such as those by a particular composer or those of a specific genre. The length of the suggestions is specified by the user. The suggestions are provided within a graphical user interface of the application and displayed as a possible direct continuation of the composition within a musical stave. If the user rejects the suggestion, additional suggestions are automatically displayed in situ. Reductive, most-probable suggestions may be offered as well as exploratory suggestions that facilitate a creative compositional interaction between user and application. Data filters enable selected aspects of a data source to be used for suggestion generation.

Abstract: A graphical user interface for music creation applications, such as score notation applications and digital audio workstations, includes multi-dimensional gestures. To enter a sound event into a musical project, a user uses an input device to select and drag a desired sound event in one or more dimensions. The relative position or rate of movement along a given dimension defines a value of a sound event parameter allocated to the given dimension. The sound event is entered into the project when the selection is released. The user inputs the gesture using a pointing device such as a mouse, stylus with a touch screen, or finger on a touch screen. Stylus dimensions mapped to sound event parameters may include, horizontal and vertical stylus tip positions, vertical and horizontal tilt of the stylus, and stylus tip pressure. Sound event parameters controlled by the gestures may include diatonic pitch, chromatic inflection, and duration.

Abstract: Automatic editing of media compositions is performed using media editing applications equipped with neural-network-based deep-learning models. The automatic editing is adapted to the practices of local users of a media editing application by training the models on a combination of media compositions previously edited by third-party media editors and media compositions edited by local users. Training data input vectors for the model comprise representative portions of a composition's raw media, and corresponding output vectors include values of parameters that define editing functions applied to the raw media to generate an edited media composition. A user interface enabling a user to adjust and monitor machine learning parameters is provided. Adaptive automatic editing may assist in the creation of video and audio compositions, as well as in the generation of musical scores.

Abstract: An operator of a digital audio workstation (DAW) application is able to assign individual tracks of a DAW session for export to specific players within a musical score of a scorewriter application. The DAW operator associates each track with a player identifier, which is retained in association with an interoperable format file generated by the export process. When the scorewriter imports such a file, it extracts the player identifier and uses it to map the track to a scored instrument. The mapping may also depend on a scorewriter arrangement of players for the instruments. The DAW operator may assign multiple tracks representing a given instrument played with different techniques to a single instrument part in a score. The playing techniques for the instruments are also associated with the tracks and may be parsed by the scorewriter to annotate the score with the corresponding notations.

Abstract: A graphical user interface of control software hosted by an audio interface device enables an operator to match a particular monitor mix with the input signals that contribute to the mix. The audio interface device receives live and non-live inputs and generates a plurality of monitor mixes. Using the graphical user interface, the operator selects a monitor mix to which a distinguishing graphical feature is assigned. In response, a graphical representation of faders representing the live and non-live inputs to the selected monitor mix automatically redisplays the faders with the distinguishing graphical feature and at an attenuation level corresponding to each input's contribution to the selected output mix. The distinguishing feature may be a color or a shading. The user interface may be used in the context of a studio recording session as well as in live performance settings.

Abstract: Low bitrate encoding of HDR-coded content with codecs that achieve compression by lowering the precision of encoding can result in significant loss of color fidelity. The problem stems from the logarithmic nature of the HDR encoding and consequent excessive quantization of color components of HDR-coded material. The loss is ameliorated by applying a saturation emphasis function that boosts color values of the HDR-coded material prior to compression. In various implementations the saturation emphasis function is a monotonically increasing function that preferentially boosts lower color values. Following compression and subsequent decompression, the color boost is reversed. In video post-production workflows proxies may be compressed to very low bitrates. For such workflows, the described methods significantly reduce codec-induced color artifacts in the decompressed proxy, thus restoring the ability of editors to make valid technical decisions based on the proxy during post-production.

Abstract: Indexing of media sources available to media composition applications, such as video editing applications and digital audio workstations, is extended to sources of an arbitrary type, including non-file type sources. Examples of such sources include devices using baseband video and audio protocols, network ports and physical devices connected via USB, Thunderbolt, etc. The sources are discovered and indexed even when not previously known to the application with the result that any available bitstream becomes available to the application. Applications access the sources via feeders, which are addressable portals through which media essence is received. The indexing methods involve the instantiation of a hierarchy of plug-in software modules by a format-handling subsystem of the application or by a media indexing service servicing one or more applications.