Abstract: Generating and storing product media can be resource intensive processes. Some systems and methods disclosed herein relate to rapidly and/or automatically generating a three-dimensional (3D) composite model to help improve the generation of product media and/or to help reduce product media storage requirements. The 3D composite model may correspond to a product bundle. According to an embodiment, an indication of items associated with a product bundle is obtained. Pre-existing 3D models corresponding to the items are then obtained and used to generate a 3D composite model. The 3D composite model includes representations of the items and may be implemented to provide product media depicting the product bundle.

Abstract: Disclosed are systems and methods for augmenting a customer image with at least one virtual object. One or more customer images depicting a customer environment are received from a customer computing device and 3D feature data for the customer images is determined. A virtual object is determined which corresponds to a desired merchant item. A positioning signal is received which corresponds to a desired location of virtual object in the customer environment, and a corresponding first location in each customer image at which to overlay the virtual object using the positioning signal is determined. An appropriate size and orientation of the virtual object is determined for each customer image based on corresponding 3D feature data, causing an overlay of an appropriately sized and oriented virtual object at the corresponding first location in customer images.

Abstract: The present disclosure relates, in part, to spatially aware media that includes three-dimensional (3D) spatial information pertaining to a real-world space. The spatially aware media may map this 3D spatial information to media such as an image, for example, to provide 3D spatial context for the media. This may allow users to more flexibly and efficiently interact with virtual content in real-world spaces that are relevant to them. According to one embodiment, spatially aware media is augmented to provide an image of a real-world space overlaid with a render of a 3D model defined relative to the 3D spatial features of the real-world space. Before augmenting the spatially aware media, a recommended position for the 3D model relative to the 3D spatial features of the real-world space may be determined based on the 3D model and/or on the spatially aware media.

Abstract: A computer-implemented method is disclosed. The method includes: receiving an indication of a first point on a three-dimensional (3D) model; determining a second point in a texture map of the 3D model that corresponds to the first point; performing segmentation of the texture map for identifying portions of the texture map that are related to the second point; and outputting a 3D representation of the identified related portions of the texture map.

Abstract: The present disclosure relates, in part, to spatially aware media that includes three-dimensional (3D) spatial information pertaining to a real-world space. The spatially aware media may map this 3D spatial information to media such as an image, for example, to provide 3D spatial context for the media. This may allow users to more flexibly and efficiently interact with virtual content in real-world spaces that are relevant to them. According to one embodiment, spatially aware media is augmented to provide an image of a real-world space overlaid with a render of a 3D model defined relative to the 3D spatial features of the real-world space. Before augmenting the spatially aware media, a recommended position for the 3D model relative to the 3D spatial features of the real-world space may be determined based on the 3D model and/or on the spatially aware media.

Abstract: Computer-implemented methods and systems including receiving feature data identifying a physical surface determined using augmented reality software and an imaging device operated by a computing device in a physical environment comprising the physical surface; determining a positioning of at least one surface item upon the physical surface based on surface dimensions determined from the feature data; and causing an augmented reality interface to be displayed by the augmented reality software on the computing device, the augmented reality interface comprising an illustrative layout of the physical surface generated using the feature data and showing the determined positioning of the at least one surface item upon the physical surface in the illustrative layout while imaging the physical surface in the physical environment.

Abstract: A merchant may operate a retail store that users are able to visit in person in order to view and purchase products. When a user visits the store, the user might not know where a desired product (“target product”) is located. Computer technology may help direct the user to the target product in real-time. In some embodiments, a model of passable areas and the location of products in the retail store may be determined, e.g. by a merchant device. In some embodiments, when the user visits the retail store, a computer generates a product recommendation, e.g. based on user-specific information, and a route in the retail store is determined for the user. In some embodiments, the route in the retail store may be determined using the model based on the target product, the user's location in the store, and one or more recommended products.

Abstract: Described herein are systems and methods for generating AR-enriched media feeds for comparing attributes of objects. A user operates an AR device to collect or extract object information in a media feed including a current object. The AR device identifies a comparison object using attributes of the current baseline object. After the comparison object has been identified, the AR device generates and presents an AR overlay in the graphical user interface that shows the selected attribute of the comparison object nearby or on top of the attribute of the current object in the real time media feed containing the current object.

Abstract: A textual description that includes a body of unstructured text is received. Using a rating model configured to output a rating based on a degree of similarity of the received textual description and each of a set of selected textual descriptions, a rating is generated based on the received textual description. The rating model to also used to generate a suggested modification of the received textual description that, when applied to the received textual descriptions, changes the rating of the received textual description. An indication of the suggested modification can be output to a user.

Abstract: A computer-implemented method is disclosed. The method includes: obtaining at least one output of a generative model based on input of a first text prompt; presenting the at least one output via a user interface; receiving, via the user interface, user selection of a desired portion of the at least one output; modifying the first text prompt based on the user selection to obtain a second text prompt; and providing the second text prompt as input to the generative model for obtaining a second output.

Abstract: Methods and systems for prompting a large language model (LLM) to generate a revised text passage with formatting are described. A text-editing instruction is received that is related to at least a portion of a text passage having at least one formatting tag. The text passage is processed to identify the at least one formatting tag in the text passage. A prompt to the LLM is generated, to cause the LLM to generate a revised text passage. The prompt includes the text-editing instruction related to at least the portion of the text passage and also includes a formatting-specific instruction to format the revised text passage using the at least one formatting tag in the revised text passage. The revised text passage is received and caused to be displayed based on the formatting tag.

Abstract: Methods and systems for indicating a resource usage parameter for prompting a large language model (LLM) are described. A user input is received, from an electronic device, for generating a prompt to a LLM. A prompt resource usage parameter is computed based on the user input. A trained resource prediction model is used to generate a predicted response resource usage parameter for a response from the LLM, based on the user input. A total resource usage parameter is computed, based on the prompt resource usage parameter and the predicted response resource usage parameter. A representation of the total resource usage parameter is communicated to the electronic device, to cause the electronic device to provide an output of the representation of the total resource usage parameter.

Abstract: The present disclosure provides a system and method for object verification. The method comprises obtaining sensor data, from one or more sensors, the sensor data including measurements of one or more physical properties of a given three-dimensional (3D) object, wherein the sensor data comprises a first measurement of the one or more physical properties from a first perspective and a second measurement of the one or more physical properties from a second perspective, the second perspective being different from the first perspective. The method further comprises comparing the sensor data with reference data, the reference data including measurements of one or more corresponding physical properties of a reference 3D object, and generating, based on the comparing, a verification signal indicating a determined match between the given 3D object and the reference 3D object.

Abstract: A computer-implemented is disclosed. The method includes: receiving a search query input; generating input enhancement data based on the search query input, the generating comprising processing the search query input using a large language model (LLM); causing to transform at least one of the search query input or the input enhancement data into a first vector embedding; and performing a search of an embedding space based on the first vector embedding.

Abstract: In virtual reality (VR) and augmented reality (AR), eye tracking may be performed to determine the user's gaze direction. The gaze direction may be used to enhance user interaction. However, when a user gazes in a particular direction, it could sometimes be the case that there are multiple items located in that gaze direction, each at a different depth. The gaze of direction alone might not be indicative of the item at which the user is looking. Therefore, in some embodiments, to try to further enhance user interaction, a gaze depth of the gaze may be determined. Some embodiments are directed to performing eye tracking to detect a gaze depth of a human's gaze and modifying a user interface (UI) responsive to a change in the gaze depth.

Abstract: Methods and systems for prompting a large language model (LLM) to process inputs from multiple user elements to generate a revised block of text are described. One or more text-editing instructions related to respective one or more selected text portions in a block of text are received. A prompt is generated for a LLM to generate a revised block of text, the prompt including at least a portion of an annotated block of text, the annotated block of text including each text-editing instruction inserted into the block of text relative to each respective selected text portion. The prompt is provided to the LLM and a revised block of text is received and outputted.

Abstract: A computer-implemented is disclosed. The method includes: obtaining identifying information for an object, the object being in a field of view of an AR device, wherein the identifying information comprises implicit signals representing contextual data associated with the object; determining that the object is associated with a first object record based on comparing the identifying information with stored identifiers associated with the first object record; and responsive to the determination that the object is associated with the first object record, presenting, via the AR device, AR content that is specific to the first object record.

Abstract: Methods and systems for automatically prompting a LLM to generate a personalized text, such as a personalized textual description, in which portions of the text are customized based on user attributes. In various examples, responsive to a request for a textual description, a user record is retrieved for a user associated with the request and one or more user attributes are obtained based on the user record. In examples, a prompt to a large language model (LLM) for generating a user-specific textual description is generated, the prompt including the one or more user attributes to include in the generated user-specific textual description and a source text. The prompt is provided to the LLM to receive a generated user-specific textual description. The generated user-specific textual description is provided for display via a user device.

Abstract: Methods and systems for prompting a large language model (LLM) to generate a description of an object with indications of any unsubstantiated information are disclosed. A prompt is generated to a LLM to generate a description of an object, where the prompt includes one or more object attributes to include in the generated description. The prompt also includes an instruction for the LLM to annotate any portions of the generated description that are, involve, and/or include unsubstantiated information according to a defined format. The prompt is provided to the LLM and the generated description is received. The generated description is parsed to identify, based on the defined format, one or more annotated portions indicating unsubstantiated information. The generated description is presented for display via a user device.

Abstract: To improve user experience when interacting with AR content within an AR environment, the AR content may be overlaid over a proxy object in a real-world space. Differences in dimension between the proxy object and the virtual model may be such that the object is larger than the virtual model, which may result in portions of the object appearing to protrude from behind the virtual model, decreasing user enjoyment. In some embodiments, an AR system for the overlay of AR content on a proxy object and concealment of the proxy object may be implemented. The system may overlay a virtual model to a proxy object, and then conceal any remaining visible portions of the proxy object from the visual field of a device displaying the AR environment. The system may overlay the virtual model so that any remaining visible portion of the proxy object is a single continuous region.