Abstract: The present disclosure describes techniques for automatically and fast generating music for videos. The techniques comprise receiving a video from a user. The video may comprise a plurality of segments of frames. Information may be extracted from the video, wherein the extracted information comprises information indicating motion speed in the video, information indicating motion saliency in the video, information indicating scene transition in the video, and timing information associated with the video. A plurality of sets of music notes matching the plurality of segments of frames may be generated based at least in part on the extracted information. A plurality of vectors corresponding to the plurality of sets of music notes may be generated. The plurality of pieces of music audio corresponding to the plurality of segments of frames may be generated based at least in part on the plurality of vectors.

Abstract: The present disclosure describes techniques for controllable music generation. The techniques comprise extracting latent vectors from unlabelled data, the unlabelled data comprising a plurality of music note sequences, the plurality of music note sequences indicating a plurality of pieces of music; clustering the latent vectors into a plurality of classes corresponding to a plurality of music styles; generating a plurality of labelled latent vectors corresponding to the plurality of music styles, each of the plurality labelled latent vectors comprising information indicating features of a corresponding music style; and generating a first music note sequence indicating a first piece of music in a particular music style among the plurality of music styles based at least in part on a particular labelled latent vector among the plurality of labelled latent vectors, the particular labelled latent vector corresponding to the particular music style.