Abstract: Embodiments described herein provide a method for autonomous agents (which may be situated, embodied agents) to learn new visual objects, and recognize the same objects within the same “runtime”/session/interaction, and be able to train a longer-term visual memory model on the object. A visual memory system stores templates of objects and compares images to these templates. An internal or external (to the agent) signal may determine which object/s to compare. The visual memory system stores new and unrecognized objects as “templates” and returns whether or not any region of the image is likely to contain a match for any of the stored templates. There is thus provided one-shot visual learning.

Abstract: Visual Prompt Tuning provides fine-tuning for transformer-based vision models. Prompt Vectors are added as additional inputs to Vision Transformer models. alongside image patches which have been linearly projected and combined with a positional embedding. The transformer architecture allows prompts to be optimized using gradient descent. without modifying or removing any of the Vision Transformer parameters. A Image Recognition System with Visual Prompt Tuning improves a pre-trained vision model by adapting the pre-trained vision model to downstream tasks by tuning the pretrained vision model using a visual prompt.