Abstract: A method for generating an audio output is described. Image inputs of interactive movements by a user captured by an image sensor are received. The interactive movements are mapped to a sequence of audio element identifiers. The sequence of audio element identifiers are processed to generate a musical sequence by performing music theory rule enforcement on the sequence of audio element identifiers. An audio output that represents the musical sequence is generated.

Abstract: There are provided methods, devices, and computer program products for processing music data. In a method, the music data is divided into a plurality of segments according to a predetermined length. A plurality of control tokens are determined for the plurality of segments based on control information associated with the plurality of segments, respectively. A plurality of sound tokens are determined for the plurality of segments based on sound information associated with the plurality of segments, respectively. A feature for the music data is obtained based on the plurality of control tokens and the plurality of sound tokens.

Abstract: In examples, a method for converting audio samples to full song arrangements is provided. The method includes receiving audio sample data, determining a melodic transcription, based on the audio sample data, and determining a sequence of music chords, based on the melodic transcription. The method further includes generating a full song arrangement, based on the sequence of music chords, and the audio sample data.

Abstract: In examples, a method for generating a remixed audio sample is provided. The method may include receiving an audio portion, obtaining metadata from the received audio portion, and analyzing the metadata and generating a symbolic music representation based on the analyzed metadata. In some examples, a selection of a style asset is received and applied to the symbolic music representation. Accordingly, a remixed audio portion may be rendered based on the stylized symbolic representation. That is, metadata associated with a song or song portion may be analyzed to identify a tempo, key, structure, chord, and/or progressions, etc., such that a remixed version of the song can be provided with customized instrumental arrangements and styles.

Abstract: Systems and methods directed to combining audio tracks are provided. More specifically, a first audio track and a second audio track are received. The first audio track is separated into a vocal component and one or more accompaniment components. The second audio track is separated into a vocal component and one or more accompaniment components. A structure of the first audio track and a structure of the second audio track are determined. The first audio track and the second audio track are aligned based on the determined structures of the tracks. The vocal component of the first audio track is stretched to match a tempo of the second audio track. The stretched vocal component of the first audio track is added to the one or more accompaniment components of the second audio track.

Abstract: Methods, systems, and storage media for generating a beatbox transcript are disclosed. Some examples may include: receiving an audio signal having a plurality of beatbox sounds, generating a spectrogram of the audio signal, processing the spectrogram of the audio signal with a neural network model trained on training samples including beatbox sounds, generating, by the neural network model a beatbox sound activation map including a plurality of activation times for a plurality of beatbox sounds, decoding the beatbox sound activation map into a beatbox transcript and providing the beatbox transcript as an output.

Abstract: Embodiments of the present disclosure relate to a music generation method, apparatus, system and storage medium. In at least some embodiments of the present disclosure, by displaying a music generation interface including a text input box, a music generation control and a music configuration item in response to an operation by a user triggering the music generation control, so that the user can input a custom text in the text input box and configure a music melody through the music configuration item, and then in response to an operation by the user triggering the music generation control, it is possible to generate a voice based on the custom text input by the user, and generate a music including the voice corresponding to the custom text based on the generated voice and the user configured music melody.

Abstract: The present disclosure relates to a song generation method, apparatus and system, and a storage medium. The song generation method includes acquiring a target lyric text input by a user; aligning the target lyric text with a singing melody of an initial song, to determine correspondence between text units in the target lyric text and notes in the singing melody, wherein the singing melody is a singing melody of initial lyrics in the initial song; performing voice synthesis on the target lyric text based on the correspondence between the text units in the target lyric text and the notes in the singing melody, to obtain a singing voice singing the target lyric text with the singing melody; and combining the singing voice with an accompaniment audio of the initial song to generate a target song.

Abstract: The present application provides a special effect processing method and apparatus. The method includes: generating an audio signal in response to a touch operation of a user in a process of playing a video; segmenting the audio signal into multiple audio frames; performing, according to attributes of the audio frames, special effect processing on a picture which is currently played in the video.

Abstract: Provided are a method and apparatus for determining an audio, an electronic device, and a storage medium. The method includes: a chord corresponding to an input audio is determined in response to receiving the input audio; at least one to-be-selected arrangement style is determined according to the chord and/or a melody attribute of the input audio; at least one to-be-selected arrangement template associated with a target arrangement style is displayed based on a trigger operation to the at least one to-be-selected arrangement style; and a target audio is played based on a trigger operation to the at least one to-be-selected arrangement template.

Abstract: An apparatus is provided that accurately estimates a point at which a musical performance is started by a player. The apparatus includes the musical performance analysis unit 32, and the musical performance analysis unit 32 obtains action data that includes a time series of feature data representing actions made by a player during a musical performance for a reference period and estimating a sound-production point based on the action data at an estimated point using a learned model L.

Abstract: The present application provides a special effect processing method and apparatus. The method includes: generating an audio signal in response to a touch operation of a user in a process of playing a video; segmenting the audio signal into multiple audio frames; performing, according to attributes of the audio frames, special effect processing on a picture which is currently played in the video.

Abstract: In examples, a method for generating a remixed audio sample is provided. The method may include receiving an audio portion, obtaining metadata from the received audio portion, and analyzing the metadata and generating a symbolic music representation based on the analyzed metadata. In some examples, a selection of a style asset is received and applied to the symbolic music representation. Accordingly, a remixed audio portion may be rendered based on the stylized symbolic representation. That is, metadata associated with a song or song portion may be analyzed to identify a tempo, key, structure, chord, and/or progressions, etc., such that a remixed version of the song can be provided with customized instrumental arrangements and styles.

Abstract: Systems and methods directed to combining audio tracks are provided. More specifically, a first audio track and a second audio track are received. The first audio track is separated into a vocal component and one or more accompaniment components. The second audio track is separated into a vocal component and one or more accompaniment components. A structure of the first audio track and a structure of the second audio track are determined. The first audio track and the second audio track are aligned based on the determined structures of the tracks. The vocal component of the first audio track is stretched to match a tempo of the second audio track. The stretched vocal component of the first audio track is added to the one or more accompaniment components of the second audio track.

Abstract: In examples, a method for converting audio samples to full song arrangements is provided. The method includes receiving audio sample data, determining a melodic transcription, based on the audio sample data, and determining a sequence of music chords, based on the melodic transcription. The method further includes generating a full song arrangement, based on the sequence of music chords, and the audio sample data.

Abstract: The present disclosure describes techniques for automatically and fast generating music for videos. The techniques comprise receiving a video from a user. The video may comprise a plurality of segments of frames. Information may be extracted from the video, wherein the extracted information comprises information indicating motion speed in the video, information indicating motion saliency in the video, information indicating scene transition in the video, and timing information associated with the video. A plurality of sets of music notes matching the plurality of segments of frames may be generated based at least in part on the extracted information. A plurality of vectors corresponding to the plurality of sets of music notes may be generated. The plurality of pieces of music audio corresponding to the plurality of segments of frames may be generated based at least in part on the plurality of vectors.

Abstract: Methods, systems, and storage media for generating a beatbox transcript are disclosed. Some examples may include: receiving an audio signal having a plurality of beatbox sounds, generating a spectrogram of the audio signal, processing the spectrogram of the audio signal with a neural network model trained on training samples including beatbox sounds, generating, by the neural network model a beatbox sound activation map including a plurality of activation times for a plurality of beatbox sounds, decoding the beatbox sound activation map into a beatbox transcript and providing the beatbox transcript as an output.

Abstract: A method for generating an audio output is described. Image inputs of interactive movements by a user captured by an image sensor are received. The interactive movements are mapped to a sequence of audio element identifiers. The sequence of audio element identifiers are processed to generate a musical sequence by performing music theory rule enforcement on the sequence of audio element identifiers. An audio output that represents the musical sequence is generated.

Abstract: Systems, methods, and computer program products for associating a media content clip(s) with other media content clip(s) having a different modality by determining first embedding vectors of media content items of a first modality, receiving a media content clip of a second modality, determining a second embedding vector of the media content clip of the second modality, ranking the first embedding vectors based on a distance between the embedding vectors and the second embedding vector, and selecting one or more of the media content items of the first modality based on the ranking, thereby pairing media content clips based on emotion.

Abstract: Systems, methods, and computer program products for associating a media content clip(s) with other media content clip(s) having a different modality by determining first embedding vectors of media content items of a first modality, receiving a media content clip of a second modality, determining a second embedding vector of the media content clip of the second modality, ranking the first embedding vectors based on a distance between the embedding vectors and the second embedding vector, and selecting one or more of the media content items of the first modality based on the ranking, thereby pairing media content clips based on emotion.