Abstract: The present disclosure describes techniques for automatically and fast generating music for videos. The techniques comprise receiving a video from a user. The video may comprise a plurality of segments of frames. Information may be extracted from the video, wherein the extracted information comprises information indicating motion speed in the video, information indicating motion saliency in the video, information indicating scene transition in the video, and timing information associated with the video. A plurality of sets of music notes matching the plurality of segments of frames may be generated based at least in part on the extracted information. A plurality of vectors corresponding to the plurality of sets of music notes may be generated. The plurality of pieces of music audio corresponding to the plurality of segments of frames may be generated based at least in part on the plurality of vectors.

Abstract: Methods, systems, and storage media for generating a beatbox transcript are disclosed. Some examples may include: receiving an audio signal having a plurality of beatbox sounds, generating a spectrogram of the audio signal, processing the spectrogram of the audio signal with a neural network model trained on training samples including beatbox sounds, generating, by the neural network model a beatbox sound activation map including a plurality of activation times for a plurality of beatbox sounds, decoding the beatbox sound activation map into a beatbox transcript and providing the beatbox transcript as an output.

Abstract: A method for generating an audio output is described. Image inputs of interactive movements by a user captured by an image sensor are received. The interactive movements are mapped to a sequence of audio element identifiers. The sequence of audio element identifiers are processed to generate a musical sequence by performing music theory rule enforcement on the sequence of audio element identifiers. An audio output that represents the musical sequence is generated.