Abstract: Systems and methods include generating synthetic videos based on a custom motion. A video generation system obtains a text prompt including an object and a custom motion token. The custom motion token represents a custom motion. The system encodes the text prompt to obtain a text embedding. Subsequently, a video generation model generates a synthetic video depicting the object performing the custom motion based on the text embedding using a video generation model.

Abstract: Techniques for learning to personalize vision-language models through meta-personalization are described. In one embodiment, one or more processing devices lock a pre-trained vision-language model (VLM) during a training phase. The processing devices train the pre-trained VLM to augment a text encoder of the pre-trained VLM with a set of general named video instances to form a meta-personalized VLM, the meta-personalized VLM to include global category features. The processing devices test the meta-personalized VLM to adapt the text encoder with a set of personal named video instances to form a personal VLM, the personal VLM comprising the global category features personalized with a set of personal instance weights to form a personal instance token associated with the user. Other embodiments are described and claimed.

Abstract: Embodiments are disclosed for an audio recommendation system trained to recommend music audio sequences for pairing with query video sequences using neural networks. In particular, in one or more embodiments, the disclosed systems and methods comprise receiving an input including a query video sequence and natural language text. The disclosed systems and methods further comprise generating a fused visual-text embedding based on a visual embedding and a text embedding corresponding to the input. The disclosed systems and methods further comprise comparing audio embeddings for music audio sequences of a music audio sequences database with the fused visual-text embedding. The disclosed systems and methods further comprise determining a music audio sequence from the music audio sequences database as the recommended music audio sequence for pairing with the query video sequence based on a similarity metric calculated between an audio embedding for the music audio sequence and the fused visual-text embedding.

Abstract: A method of identifying a user-specified object contained in one or more images of a plurality of images that comprises the steps of defining regions of objects in the images, and computing a vector in respect of each of the regions based on the appearance of the respective region. The vector comprises a descriptor. The method further comprises vector quantizing the descriptors into clusters, storing the clusters as an index with the images in which they occur, defining regions of the user-specified object, computing a vector in respect of each of said regions based on the appearance of the regions, and vector quantizing the descriptors into one or more clusters. The index is searched and the clusters are compared with the contents of the index to identify which of the images contain the clusters so as to return the images containing the user-defined object.