Abstract: The present disclosure relates, in part, to spatially aware media that includes three-dimensional (3D) spatial information pertaining to a real-world space. The spatially aware media may map this 3D spatial information to media such as an image, for example, to provide 3D spatial context for the media. This may allow users to more flexibly and efficiently interact with virtual content in real-world spaces that are relevant to them. According to one embodiment, spatially aware media is augmented to provide an image of a real-world space overlaid with a render of a 3D model defined relative to the 3D spatial features of the real-world space. Before augmenting the spatially aware media, a recommended position for the 3D model relative to the 3D spatial features of the real-world space may be determined based on the 3D model and/or on the spatially aware media.

Abstract: A merchant may operate a retail store that users are able to visit in person in order to view and purchase products. When a user visits the store, the user might not know where a desired product (“target product”) is located. Computer technology may help direct the user to the target product in real-time. In some embodiments, a model of passable areas and the location of products in the retail store may be determined, e.g. by a merchant device. In some embodiments, when the user visits the retail store, a computer generates a product recommendation, e.g. based on user-specific information, and a route in the retail store is determined for the user. In some embodiments, the route in the retail store may be determined using the model based on the target product, the user's location in the store, and one or more recommended products.

Abstract: Systems and methods described herein allow a customer to employ AR/VR software to generate virtual representations of physical spaces (e.g., house) and sub-spaces (e.g., living room) to preview virtual objects situated in AR/VR virtual environments. A commerce system (or mobile app associated with the commerce system) may generate virtualized environments representing a physical space (e.g., house, apartment) and regions (e.g., living room, kitchen) based on source images uploaded to or otherwise captured by the commerce system. The end-user may operate the software on a client device and interacts with VR or AR presentations of the virtual environment using a voice-based interface recognized by the software. For example, the end-user may say the name of room (region) or an object and the system retrieves data of the identified room or an appropriate room, such as virtual representations of furniture or objects situated in the room.

Abstract: When a person visits a physical retail store, the merchant often does not have enough information about the person to make meaningful product recommendations. Also, a physical retail store typically has products physically distributed throughout the store. It may be desirable to have some sort of relationship between where the customer is and the location of the product being recommended. In some embodiments, when a person visits the store, a computer determines an identity of the person and generates a product recommendation based on user-specific information for that person. In some embodiments, generating the product recommendation includes detecting that a field of view of a camera of the device has changed, and in response determining a plurality of products within or proximate to a current field of view of the camera. At least one of the plurality of products is then identified as the recommended product.

Abstract: Systems and methods described herein allow a customer to employ AR/VR software to generate virtual representations of physical spaces (e.g., house) and sub-spaces (e.g., living room) to preview virtual objects situated in AR/VR virtual environments. A commerce system (or mobile app associated with the commerce system) may generate virtualized environments representing a physical space (e.g., house, apartment) and regions (e.g., living room, kitchen) based on source images uploaded to or otherwise captured by the commerce system. The end-user may operate the software on a client device and interacts with VR or AR presentations of the virtual environment using a voice-based interface recognized by the software. For example, the end-user may say the name of room (region) or an object and the system retrieves data of the identified room or an appropriate room, such as virtual representations of furniture or objects situated in the room.

Abstract: Systems and methods for generating AR content to virtually alter a real-world item are provided. The real-world item may be a deformable item including one or more flexible surfaces. According to an embodiment, a method includes determining the relative positions of a plurality of points on the real-world item. Based on the relative positions of the plurality of points, a 3D model of a virtual item is distorted to produce a distorted 3D model of the virtual item. The distorted 3D model may have substantially the same shape as the real-world item. AR content is then generated based on the distorted three-dimensional model.

Abstract: Systems and methods are provided for the generation of augmented reality (AR) content that provides a shared AR experience involving multiple users. Shared AR experiences can improve the communication and collaboration between multiple simultaneous users. According to an embodiment, AR content is generated for a first user in a shared AR experience. The AR content includes at least one of a render of a model, a virtual representation of a second user in the shared AR experience, a virtual representation of a user interaction in the shared AR experience, and spatial audio content. Modifications to a shared AR experience are also provided. These modifications may be initiated based on instructions from one user and be reflected in the AR content generated for multiple users.

Abstract: When a person visits a physical retail store, the merchant often does not have enough information about the person to make meaningful product recommendations. Also, a physical retail store typically has products physically distributed throughout the store. It may be desirable to have some sort of relationship between where the customer is and the location of the product being recommended. In some embodiments, when a person visits the store, a computer determines an identity of the person and generates a product recommendation based on user-specific information for that person. In some embodiments, generating the product recommendation includes detecting that a field of view of a camera of the device has changed, and in response determining a plurality of products within or proximate to a current field of view of the camera. At least one of the plurality of products is then identified as the recommended product.

Abstract: Systems and methods described herein allow a customer to employ AR/VR software to generate virtual representations of physical spaces (e.g., house) and sub-spaces (e.g., living room) to preview virtual objects situated in AR/VR virtual environments. A commerce system (or mobile app associated with the commerce system) may generate virtualized environments representing a physical space (e.g., house, apartment) and regions (e.g., living room, kitchen) based on source images uploaded to or otherwise captured by the commerce system. The end-user may operate the software on a client device and interacts with VR or AR presentations of the virtual environment using a voice-based interface recognized by the software. For example, the end-user may say the name of room (region) or an object and the system retrieves data of the identified room or an appropriate room, such as virtual representations of furniture or objects situated in the room.

Abstract: Systems and methods are provided for the generation of augmented reality (AR) content that provides a shared AR experience involving multiple users. Shared AR experiences can improve the communication and collaboration between multiple simultaneous users. According to an embodiment, AR content is generated for a first user in a shared AR experience. The AR content includes at least one of a render of a model, a virtual representation of a second user in the shared AR experience, a virtual representation of a user interaction in the shared AR experience, and spatial audio content. Modifications to a shared AR experience are also provided. These modifications may be initiated based on instructions from one user and be reflected in the AR content generated for multiple users.

Abstract: The consistent use of lighting in different instances of digital media may help ensure that objects are depicted in a similar manner in the digital media. However, in some cases, a three-dimensional (3D) model may depict an object under lighting conditions that differ from the lighting conditions depicted in other digital media. The present disclosure provides systems and methods for generating 3D models to include lighting that is consistent with the lighting used in other digital media. According to an embodiment, a lighting template is determined based on digital media depicting a first physical object. A modified 3D model of a second physical object is then generated based on the lighting template to light the second physical object according to the lighting template.

Abstract: Systems and methods for generating AR content to virtually alter a real-world item are provided. The real-world item may be a deformable item including one or more flexible surfaces. According to an embodiment, a method includes determining the relative positions of a plurality of points on the real-world item. Based on the relative positions of the plurality of points, a 3D model of a virtual item is distorted to produce a distorted 3D model of the virtual item. The distorted 3D model may have substantially the same shape as the real-world item. AR content is then generated based on the distorted three-dimensional model.

Abstract: Systems and methods are provided for the generation of augmented reality (AR) content that provides a shared AR experience involving multiple users. Shared AR experiences can improve the communication and collaboration between multiple simultaneous users. According to an embodiment, AR content is generated for a first user in a shared AR experience. The AR content includes at least one of a render of a model, a virtual representation of a second user in the shared AR experience, a virtual representation of a user interaction in the shared AR experience, and spatial audio content. Modifications to a shared AR experience are also provided. These modifications may be initiated based on instructions from one user and be reflected in the AR content generated for multiple users.

Abstract: The present disclosure relates, in part, to spatially aware media that includes three-dimensional (3D) spatial information pertaining to a real-world space. The spatially aware media may map this 3D spatial information to media such as an image, for example, to provide 3D spatial context for the media. This may allow users to more flexibly and efficiently interact with virtual content in real-world spaces that are relevant to them. According to one embodiment, spatially aware media is augmented to provide an image of a real-world space overlaid with a render of a 3D model defined relative to the 3D spatial features of the real-world space. Before augmenting the spatially aware media, a recommended position for the 3D model relative to the 3D spatial features of the real-world space may be determined based on the 3D model and/or on the spatially aware media.

Abstract: Systems and methods are provided for the generation of augmented reality (AR) scenery for a physical item. The AR scenery allows a customer to interact with the physical item in the real-world, while also allowing the customer to view the physical item in a virtual setting that may be tailored to the customer and/or to the physical item. According to an embodiment, an AR scene is generated for a physical item. The AR scene includes computer-generated scenery and at least a portion of an image of the physical item, where the computer-generated scenery is based on information associated with the user and/or the physical item. The AR scene is then displayed on a device.

Abstract: A merchant may operate a retail store that users are able to visit in person in order to view and purchase products. When a user visits the store, the user might not know where a desired product (“target product”) is located. Computer technology may help direct the user to the target product in real-time. In some embodiments, a model of passable areas and the location of products in the retail store may be determined, e.g. by a merchant device. In some embodiments, when the user visits the retail store, a computer generates a product recommendation, e.g. based on user-specific information, and a route in the retail store is determined for the user. In some embodiments, the route in the retail store may be determined using the model based on the target product, the user's location in the store, and one or more recommended products.

Abstract: High-fidelity three-dimensional (3D) models and other high-fidelity digital media that depict objects with a high-level of detail may be computationally demanding to display on some devices. According to some embodiments of the present disclosure, digital media may be supplemented with one or more 3D models to improve the overall level of detail provided by the digital media without excessively increasing computational requirements. An example computer-implemented method includes instructing a user device to display digital media depicting an object, receiving an indication selecting a region of the depicted object, and instructing the user device to display a 3D model corresponding to the selected region of the depicted object, where the 3D model is different from the digital media.

Abstract: Systems and methods are provided for the generation of augmented reality (AR) scenery for a physical item. The AR scenery allows a customer to interact with the physical item in the real-world, while also allowing the customer to view the physical item in a virtual setting that may be tailored to the customer and/or to the physical item. According to an embodiment, an AR scene is generated for a physical item. The AR scene includes computer-generated scenery and at least a portion of an image of the physical item, where the computer-generated scenery is based on information associated with the user and/or the physical item. The AR scene is then displayed on a device.

Abstract: The present disclosure relates, in part, to spatially aware media that includes three-dimensional (3D) spatial information pertaining to a real-world space. The spatially aware media may map this 3D spatial information to media such as an image, for example, to provide 3D spatial context for the media. This may allow users to more flexibly and efficiently interact with virtual content in real-world spaces that are relevant to them. According to one embodiment, spatially aware media is augmented to provide an image of a real-world space overlaid with a render of a 3D model defined relative to the 3D spatial features of the real-world space. Before augmenting the spatially aware media, a recommended position for the 3D model relative to the 3D spatial features of the real-world space may be determined based on the 3D model and/or on the spatially aware media.

Abstract: A merchant may operate a retail store that users are able to visit in person in order to view and purchase products. When a user visits the store, the user might not know where a desired product (“target product”) is located. Computer technology may help direct the user to the target product in real-time. In some embodiments, a model of passable areas and the location of products in the retail store may be determined, e.g. by a merchant device. In some embodiments, when the user visits the retail store, a computer generates a product recommendation, e.g. based on user-specific information, and a route in the retail store is determined for the user. In some embodiments, the route in the retail store may be determined using the model based on the target product, the user's location in the store, and one or more recommended products.