Abstract: A method for generating an audio output is described. Image inputs of interactive movements by a user captured by an image sensor are received. The interactive movements are mapped to a sequence of audio element identifiers. The sequence of audio element identifiers are processed to generate a musical sequence by performing music theory rule enforcement on the sequence of audio element identifiers. An audio output that represents the musical sequence is generated.

Abstract: Embodiments of the disclosure relate to a method, apparatus, device and storage medium for generating music content. The method provided herein includes: obtaining a set of tokens generated based on input information; providing the set of tokens to a target model to generate a plurality of encoded representations corresponding to a plurality of chunks, wherein a target encoded representation corresponding to a first chunk is generated based on a first set of attention parameters associated with a second chunk, the second chunk is earlier in time than the first chunk; and generating target music content by decoding the plurality of encoded representations.

Abstract: Embodiments of the disclosue relate to a method, apparatus, device, and storage medium of training a music compression system. The method includes: obtaining a first encoded representation associated with training music content; processing the first encoded representation with the discrete encoder to generate a first set of discrete features corresponding to first music data and a second set of discrete features corresponding to second music data; decoding the first set of discrete features with the first discrete decoder to obtain a first audio feature corresponding to the first music data, and decoding the second set of discrete features with the second discrete decoder to obtain a second audio feature corresponding to the second music data; and determining a training loss based on the first audio feature, the second audio feature and the training music content, and adjusting parameters of the discrete encoder and the discrete decoders based on training loss.

Abstract: A method, an apparatus, a device, and a storage medium for training a model are provided. First audio content associated with vocal content is extracted from a music sample. First annotation information is generated based on the first audio content, and the first annotation information includes text content corresponding to the first audio content and first melody information of the first audio content. A first training sequence is constructed based on the text content and the first melody information. The first training sequence is input to a music generation model to generate a first set of music encoded representations. The music generation model is trained based on the first set of music encoded representations and a second set of music encoded representations of the music sample.

Abstract: A method, an apparatus, a device, and a storage medium for training a model are provided. The method includes: constructing a set of candidate lyrics content based on reference lyrics content, each candidate lyrics content including at least one paragraph in the reference lyrics content; determining target lyrics content satisfying a predetermined requirement from the set of candidate lyrics content based on evaluation information of the set of candidate lyrics content; generating description information corresponding to the target lyrics content, the description information indicating a plurality of attributes of the target lyrics content; constructing a set of prompts corresponding to the target lyrics content based on the description information; and training a lyrics generation model based on the set of prompts and the target lyrics content.

Abstract: A method, an apparatus, a device and a storage medium for training a generation model are provided. The method provided by the disclosure includes: obtaining first audio content corresponding to a first timbre and second audio content corresponding to a second timbre; processing the first audio content and the second audio content with a first generation model to generate third audio content; providing the third audio content and a first portion of the second audio content to a second generation model to generate a first audio feature; and training the second generation model based on the first audio feature and a second audio feature corresponding to the second audio content.

Abstract: In examples, a method for converting audio samples to full song arrangements is provided. The method includes receiving audio sample data, determining a melodic transcription, based on the audio sample data, and determining a sequence of music chords, based on the melodic transcription. The method further includes generating a full song arrangement, based on the sequence of music chords, and the audio sample data.

Abstract: In examples, a method for generating a remixed audio sample is provided. The method may include receiving an audio portion, obtaining metadata from the received audio portion, and analyzing the metadata and generating a symbolic music representation based on the analyzed metadata. In some examples, a selection of a style asset is received and applied to the symbolic music representation. Accordingly, a remixed audio portion may be rendered based on the stylized symbolic representation. That is, metadata associated with a song or song portion may be analyzed to identify a tempo, key, structure, chord, and/or progressions, etc., such that a remixed version of the song can be provided with customized instrumental arrangements and styles.

Abstract: Systems and methods directed to combining audio tracks are provided. More specifically, a first audio track and a second audio track are received. The first audio track is separated into a vocal component and one or more accompaniment components. The second audio track is separated into a vocal component and one or more accompaniment components. A structure of the first audio track and a structure of the second audio track are determined. The first audio track and the second audio track are aligned based on the determined structures of the tracks. The vocal component of the first audio track is stretched to match a tempo of the second audio track. The stretched vocal component of the first audio track is added to the one or more accompaniment components of the second audio track.

Abstract: Methods, systems, and storage media for generating a beatbox transcript are disclosed. Some examples may include: receiving an audio signal having a plurality of beatbox sounds, generating a spectrogram of the audio signal, processing the spectrogram of the audio signal with a neural network model trained on training samples including beatbox sounds, generating, by the neural network model a beatbox sound activation map including a plurality of activation times for a plurality of beatbox sounds, decoding the beatbox sound activation map into a beatbox transcript and providing the beatbox transcript as an output.

Abstract: The present disclosure relates to a music generation method, apparatus and system, and storage medium. In an embodiment of the present disclosure: obtaining text information, and converting the text information into a corresponding voice audio; obtaining an initial music audio, wherein the initial music audio comprises a music key point, and music characteristics of the initial music audio have a sudden change at the position of an audio key point; and on the basis of the position of the music key point, synthesizing the voice audio and the initial music audio to obtain a target music audio. In the target music audio, the voice audio appears at the position of the music key point of the initial music audio. Thus, a music audio is generated from text information, and the user can customize the content of the text information and customize the initial music audio.

Abstract: The present disclosure describes techniques for differentiable wavetable synthesizer. The techniques comprise extracting features from a dataset of sounds, wherein the features comprise at least timbre embedding; input the features to the first machine learning model, wherein the first machine learning model is configured to extract a set of N×L learnable parameters, N represents a number of wavetables, and L represents a wavetable length; outputting a plurality of wavetables, wherein each of plurality of wavetables comprises a waveform associated with a unique timbre, the plurality of wavetables form a dictionary, and the plurality of wavetables are portable to perform audio-related tasks. Finally, the said wavetables are used to initialize another machine learning model so as to help reduce computational complexity of an audio synthesis obtained as output of the another machine learning model.

Abstract: Embodiments of the present disclosure relate to a music generation method, apparatus, system and storage medium. In at least some embodiments of the present disclosure, by displaying a music generation interface including a text input box, a music generation control and a music configuration item in response to an operation by a user triggering the music generation control, so that the user can input a custom text in the text input box and configure a music melody through the music configuration item, and then in response to an operation by the user triggering the music generation control, it is possible to generate a voice based on the custom text input by the user, and generate a music including the voice corresponding to the custom text based on the generated voice and the user configured music melody.

Abstract: The present disclosure relates to a song generation method, apparatus and system, and a storage medium. The song generation method includes acquiring a target lyric text input by a user; aligning the target lyric text with a singing melody of an initial song, to determine correspondence between text units in the target lyric text and notes in the singing melody, wherein the singing melody is a singing melody of initial lyrics in the initial song; performing voice synthesis on the target lyric text based on the correspondence between the text units in the target lyric text and the notes in the singing melody, to obtain a singing voice singing the target lyric text with the singing melody; and combining the singing voice with an accompaniment audio of the initial song to generate a target song.

Abstract: The present application provides a special effect processing method and apparatus. The method includes: generating an audio signal in response to a touch operation of a user in a process of playing a video; segmenting the audio signal into multiple audio frames; performing, according to attributes of the audio frames, special effect processing on a picture which is currently played in the video.

Abstract: A music generation system is provided comprising a processor and a memory operatively coupled to the processor and storing a rhythm template database comprising a plurality of rhythm templates, and a music generation program stored in the memory and executed by the processor to be configured to receive a user input of lyrics, identify a plurality of syllables in the lyrics, determine a syllable pattern in the identified plurality of syllables, match the syllable pattern to a selected rhythm template of the plurality of rhythm templates, generate a melody based on the selected rhythm template, generate a music file encoding the melody and the lyrics, and output the music file encoding the melody and the lyrics.

Abstract: The present application provides a special effect processing method and apparatus. The method includes: generating an audio signal in response to a touch operation of a user in a process of playing a video; segmenting the audio signal into multiple audio frames; performing, according to attributes of the audio frames, special effect processing on a picture which is currently played in the video.

Abstract: In examples, a method for generating a remixed audio sample is provided. The method may include receiving an audio portion, obtaining metadata from the received audio portion, and analyzing the metadata and generating a symbolic music representation based on the analyzed metadata. In some examples, a selection of a style asset is received and applied to the symbolic music representation. Accordingly, a remixed audio portion may be rendered based on the stylized symbolic representation. That is, metadata associated with a song or song portion may be analyzed to identify a tempo, key, structure, chord, and/or progressions, etc., such that a remixed version of the song can be provided with customized instrumental arrangements and styles.

Abstract: Systems and methods directed to combining audio tracks are provided. More specifically, a first audio track and a second audio track are received. The first audio track is separated into a vocal component and one or more accompaniment components. The second audio track is separated into a vocal component and one or more accompaniment components. A structure of the first audio track and a structure of the second audio track are determined. The first audio track and the second audio track are aligned based on the determined structures of the tracks. The vocal component of the first audio track is stretched to match a tempo of the second audio track. The stretched vocal component of the first audio track is added to the one or more accompaniment components of the second audio track.

Abstract: In examples, a method for converting audio samples to full song arrangements is provided. The method includes receiving audio sample data, determining a melodic transcription, based on the audio sample data, and determining a sequence of music chords, based on the melodic transcription. The method further includes generating a full song arrangement, based on the sequence of music chords, and the audio sample data.