Abstract: Systems and methods are provided for selectively storing and retrieving video data by converting video segments into textual descriptions associated with key visual features and incrementally building a vector database of feature embeddings. The feature embeddings can be used in combination with text-to-video generation models to reconstruct video segments on demand. Selective storage of feature embeddings enables optimized, accurate, and contextually relevant text-to-text and text-to-video generation and video searching functionalities.

Abstract: Systems and methods in accordance with embodiments of the invention are configured to decode images containing an image of a scene and a corresponding depth map. A depth-based effect is applied to the image to generate a synthetic image of the scene. The synthetic image can be encoded into a new image file that contains metadata associated with the depth-based effect. In many embodiments, the original decoded image has a different depth-based effect applied to it with respect to the synthetic image.

Abstract: Systems and methods for enhancing a live scene to be captured by a camera of a user device based on one or more best matched reference images that include similar subject matter and attributes of the live scene are disclosed. A preview of a live scene is displayed on the user device. Attributes of the live scene are identified. The live scene is segmented separately into a foreground and a background. Vector representations of the attributes of the segmented live scene are calculated and user device parameters are obtained, and both are used to identify reference images that include the attributes of the live scene. The identified reference images are displayed on the user device and a selection is made. The user device parameters are configured based on the selection made to capture the live scene to obtain similar effects of the selected reference image.

Abstract: Systems in accordance with embodiments of the invention can perform parallax detection and correction in images captured using array cameras. Due to the different viewpoints of the cameras, parallax results in variations in the position of objects within the captured images of the scene. Methods in accordance with embodiments of the invention provide an accurate account of the pixel disparity due to parallax between the different cameras in the array, so that appropriate scene-dependent geometric shifts can be applied to the pixels of the captured images when performing super-resolution processing. In a number of embodiments, generating depth estimates considers the similarity of pixels in multiple spectral channels. In certain embodiments, generating depth estimates involves generating a confidence map indicating the reliability of depth estimates.

Abstract: Systems and methods are described for inputting text input to a trained machine learning model; generating, using the trained machine learning model and based on the text input, a single-layer image comprising a plurality of objects; segmenting the single-layer image to generate a plurality of images, each image of the plurality of images comprising a depiction of a respective object of the plurality of objects of the single-layer image; extracting, from the text input, a portion of the text input describing a background portion of the single-layer image; generating, using the trained machine learning model and based on the extracted portion of the text input, a background image; and generating the multi-layer image based on the plurality of images and the background image.

Abstract: Systems and methods for implementing array cameras configured to perform super-resolution processing to generate higher resolution super-resolved images using a plurality of captured images and lens stack arrays that can be utilized in array cameras are disclosed. An imaging device in accordance with one embodiment of the invention includes at least one imager array, and each imager in the array comprises a plurality of light sensing elements and a lens stack including at least one lens surface, where the lens stack is configured to form an image on the light sensing elements, control circuitry configured to capture images formed on the light sensing elements of each of the imagers, and a super-resolution processing module configured to generate at least one higher resolution super-resolved image using a plurality of the captured images.

Abstract: Systems in accordance with embodiments of the invention can perform parallax detection and correction in images captured using array cameras. Due to the different viewpoints of the cameras, parallax results in variations in the position of objects within the captured images of the scene. Methods in accordance with embodiments of the invention provide an accurate account of the pixel disparity due to parallax between the different cameras in the array, so that appropriate scene-dependent geometric shifts can be applied to the pixels of the captured images when performing super-resolution processing. In a number of embodiments, generating depth estimates considers the similarity of pixels in multiple spectral channels. In certain embodiments, generating depth estimates involves generating a confidence map indicating the reliability of depth estimates.

Abstract: A region of interest is determined based on metadata associated with an image. The region of interest is then sliced into a plurality of slices. POI information is retrieved for each respective slice. An overlay for at least one POI is then generated for display over the image.

Abstract: Systems and methods are provided for enabling improved image capture at a computing device comprising a plurality of cameras. First and second capture streams, from respective first and second cameras of a computing device, are received at the computing device, wherein the first and second cameras face in different directions. A region of the first capture stream to include as an overlay over a portion of the second capture stream is identified. It is determined that a combined frame comprising a frame from the second capture stream with an overlay from the region of the first capture stream meets threshold criterion based on image component analysis, and, in response to the determining, an image based on the combined frame is stored in a non-transitory memory.

Abstract: Systems and methods for estimating depth from projected texture using camera arrays are described. A camera array includes a conventional camera and at least one two-dimensional array of cameras, where the conventional camera has a higher resolution than the cameras in the at least one two-dimensional array of cameras, an illumination system configured to illuminate a scene with a projected texture, where an image processing pipeline application directs the processor to: utilize the illumination system controller application to control the illumination system to illuminate a scene with a projected texture, capture a set of images of the scene illuminated with the projected texture, and determining depth estimates for pixel locations in an image from a reference viewpoint using at least a subset of the set of images.

Abstract: Systems and methods for calibrating an array camera are disclosed. Systems and methods for calibrating an array camera in accordance with embodiments of this invention include the capturing of an image of a test pattern with the array camera such that each imaging component in the array camera captures an image of the test pattern. The image of the test pattern captured by a reference imaging component is then used to derive calibration information for the reference component. A corrected image of the test pattern for the reference component is then generated from the calibration information and the image of the test pattern captured by the reference imaging component. The corrected image is then used with the images captured by each of the associate imaging components associated with the reference component to generate calibration information for the associate imaging components.

Abstract: In an embodiment, a 3D facial modeling system includes a plurality of cameras configured to capture images from different viewpoints, a processor, and a memory containing a 3D facial modeling application and parameters defining a face detector, wherein the 3D facial modeling application directs the processor to obtain a plurality of images of a face captured from different viewpoints using the plurality of cameras, locate a face within each of the plurality of images using the face detector, wherein the face detector labels key feature points on the located face within each of the plurality of images, determine disparity between corresponding key feature points of located faces within the plurality of images, and generate a 3D model of the face using the depth of the key feature points.

Abstract: Systems and methods are described for generating, using the first trained machine learning model and based on text input, a single-layer image comprising a plurality of objects; generating a plurality of masks associated with the plurality of objects; determining a plurality of attributes associated with the plurality of objects; generating, using a second trained machine learning model, a plurality of textual descriptions respectively corresponding to the plurality of objects; inputting the plurality of textual descriptions, and the plurality of attributes, to the first trained machine learning model; generating, using the first trained machine learning model, a plurality of images respectively corresponding to the plurality of textual descriptions; and generating the multi-layer image by combining the plurality of images and by using the plurality of masks, wherein the plurality of images respectively correspond to a plurality of layers of the multi-layer image.

Abstract: Systems and methods for estimating depth from projected texture using camera arrays are described. A camera array includes a conventional camera and at least one two-dimensional array of cameras, where the conventional camera has a higher resolution than the cameras in the at least one two-dimensional array of cameras, an illumination system configured to illuminate a scene with a projected texture, where an image processing pipeline application directs the processor to: utilize the illumination system controller application to control the illumination system to illuminate a scene with a projected texture, capture a set of images of the scene illuminated with the projected texture, and determining depth estimates for pixel locations in an image from a reference viewpoint using at least a subset of the set of images.

Abstract: Methods, systems, and devices are described herein for providing an extended reality (XR) experience capable of both augmented reality (AR) and virtual reality (VR) functions. The XR device comprises transparent components enabling optical see-through (OST); the device comprising a transparent microLED display and a transparent image sensor together with a meta-lens. The XR OST device can switch between a fully immersive VR mode and an AR mode by controlling the display's transparency. Light from the exterior real-world scene is collected by the meta-lens on a wide field of view (FOV) and focused onto the transparent image sensor to detect objects in the real-world environment.

Abstract: The present disclosure is directed to systems and methods to enhance the process of creating an artificial intelligence (AI) generated content or content items, such as images, text, video, sounds, etc., using a text or other suitable prompt, such as via voice input. The systems and methods disclosed provide streamlined content generation with, e.g., reduced processing power and computing time. In an embodiment the systems and methods receive a prompt for generating a first content item using a generative artificial intelligence (AI) model and retrieve, based on the prompt, a collection of matching content items. The systems and methods may then receive input selecting one of the content items from the collection and identify a prompt used to generate the selected content item. The systems and methods may then merge using a trained natural language processing model, the received prompt with the prompt of the selected content item to create a third prompt.

Abstract: Systems and methods are described for correcting rolling shutter effect for images captured by a camera. In some embodiments, an adjustable transparency layer is placed between an image sensor and a camera lens, where the adjustable transparency layer is controlled to become transparent and then opaque in order to produce a single concurrent effective exposure time, reducing rolling shutter effect.

Abstract: Systems and methods for hybrid depth regularization in accordance with various embodiments of the invention are disclosed. In one embodiment of the invention, a depth sensing system comprises a plurality of cameras; a processor; and a memory containing an image processing application. The image processing application may direct the processor to obtain image data for a plurality of images from multiple viewpoints, the image data comprising a reference image and at least one alternate view image; generate a raw depth map using a first depth estimation process, and a confidence map; and generate a regularized depth map. The regularized depth map may be generated by computing a secondary depth map using a second different depth estimation process; and computing a composite depth map by selecting depth estimates from the raw depth map and the secondary depth map based on the confidence map.

Abstract: Systems and methods for implementing array cameras configured to perform super-resolution processing to generate higher resolution super-resolved images using a plurality of captured images and lens stack arrays that can be utilized in array cameras are disclosed. An imaging device in accordance with one embodiment of the invention includes at least one imager array, and each imager in the array comprises a plurality of light sensing elements and a lens stack including at least one lens surface, where the lens stack is configured to form an image on the light sensing elements, control circuitry configured to capture images formed on the light sensing elements of each of the imagers, and a super-resolution processing module configured to generate at least one higher resolution super-resolved image using a plurality of the captured images.

Abstract: Systems and methods are provided for improving image item editing. An image item is selected at a computing device and with an editing application, and a preferred editing option to apply to the image item is identified via a user profile. The preferred editing option is determined based on historic editing actions for a plurality of different image items. An icon for applying the preferred editing option to the image item is generated for display in a user interface of the editing application. User input associated with the icon is received, and the preferred editing option is applied to the image item.