Abstract: Some examples of the disclosure are directed to systems and methods for presenting one or more user interface elements including suggested content based on detection of physical objects in a computer-generated environment. In some examples, the electronic device captures one or more images of a physical environment, including a first object (e.g., a fillable form). In some examples, in accordance with detecting, in the one or more images, a first region (e.g., a first fillable field) of the first object and that one or more first criteria are satisfied, the electronic device displays a first user interface element including first suggested text, wherein the first user interface element is displayed at a first location based on the first region.

Abstract: A method includes identifying a physical object within an environment using a camera of an electronic device, obtaining a content item to be output for display by the electronic device, determining a content position and a content orientation for the content item relative to the physical object, and displaying a representation of the content item using the electronic device so that, from a perspective of a user using the electronic device, the content item replaces the physical object in the environment. The content item is selected based on identifying the physical object.

Abstract: In one implementation, a method of storing object information in association with contextual information is performed at a device including an image sensor, one or more processors, and non-transitory memory. The method includes capturing, using the image sensor, an image of an environment. The method includes detecting a user engagement with an object in the environment based on the image of the environment. The method includes, in response to detecting the user engagement with the object, obtaining information regarding the object, obtaining contextual information, and storing, in a database, an entry including the information regarding the object in association with the contextual information.

Abstract: A method includes obtaining an image of a machine-readable data representation that is located on a physical object using a camera of an electronic device. The machine-readable data representation includes an encoded form of a data value. The method further includes decoding the machine-readable data representation to determine the data value, whereby the data value includes a content identifier and a content source identifier. The method also includes selecting a content source based on the content source identifier, obtaining a content item and content location information based on the content identifier from the content source, determining a content position and a content orientation for the content item relative to the physical object based on the content location information, and displaying a representation of the content item using the electronic device according to the content position and the content orientation.

Abstract: In one implementation, a method of displaying a plurality of glints method includes capturing an image of an environment including a plurality of objects. The method includes detecting, in the image of the environment, a plurality of actionable items respectively associated with a plurality of actions. The method includes relating the actionable items to respective ones of the plurality of objects. The method includes displaying a plurality of group user interface elements in respective association with the plurality of objects. The method includes detecting user input directed to a particular group user interface element of the plurality of group user interface elements associated with a particular object of the plurality of objects. The method includes displaying a plurality of user interface elements in respective association with those of the plurality of actionable items related to the particular object.

Abstract: Systems and methods for facilitating selection of a target object required by a request from a user from a set of candidate objects include identifying the set of candidate objects in a gaze region corresponding to a gaze of the user and generating a graphical user interface allowing a user to select the target object from the set of candidate objects.

Abstract: A head-mounted device may be used to perform a visual search on a physical environment around the head-mounted device. A user may wish to visually search one out of multiple physical objects in the physical environment. To clearly show the user which physical object was the target of a visual search, the head-mounted device may present a thumbnail of the candidate physical object on a display. The thumbnail may be cropped and/or zoomed using an image from a camera on the head-mounted device. By displaying a thumbnail that is taken by a camera on the head-mounted device, the thumbnail will directly match the physical object in the user's physical environment, eliminating ambiguity regarding which physical object is the subject of the visual search.

Abstract: In one implementation, a method of displaying text suggestions is performed at a device including an input device, a display, an image sensor, one or more processors, and non-transitory memory. The method includes obtaining, using the image sensor, one or more images of a physical environment. The method includes obtaining one or more semantic labels associated with the physical environment based on the one or more images of the physical environment. The method includes receiving, via the input device, text. The method includes determining one or more text suggestions based on the one or more semantic labels associated with the physical environment and the text. The method includes displaying, on the display, the one or more text suggestions.

Abstract: A method includes obtaining an image of a machine-readable data representation that is located on a physical object using a camera of an electronic device. The machine-readable data representation includes an encoded form of a data value. The method further includes decoding the machine-readable data representation to determine the data value, whereby the data value includes a content identifier and a content source identifier. The method also includes selecting a content source based on the content source identifier, obtaining a content item and content location information based on the content identifier from the content source, determining a content position and a content orientation for the content item relative to the physical object based on the content location information, and displaying a representation of the content item using the electronic device according to the content position and the content orientation.

Abstract: A method includes determining a device location of an electronic device, transmitting a request to a content source, the request including the device location of the electronic device, and receiving, from the content source in response to the request, a content item that is associated with display location information that describes a content position for the content item relative to a physical environment. The content item is selected by the content source based on the content position for the content item being within an area that is defined based on the device location. The method also includes displaying a representation of the content item as part of a computer-generated reality scene in which the representation of the content item is positioned relative to the physical environment according to the content position for the content item from the display location information for the content item.

Abstract: In one implementation, a method of presenting virtual content is performed by a device including an image sensor, one or more processors, and non-transitory memory. The method includes obtaining, using the image sensor, an image of a physical environment. The method includes detecting, in the image of the physical environment, machine-readable content associated with an object. The method includes determining an object type of the object. The method includes obtaining virtual content based on a search query creating using the machine-readable content and the object type. The method includes displaying the virtual content.

Abstract: A method includes determining a device location of an electronic device, transmitting a request to a content source, the request including the device location of the electronic device, and receiving, from the content source in response to the request, a content item that is associated with display location information that describes a content position for the content item relative to a physical environment. The content item is selected by the content source based on the content position for the content item being within an area that is defined based on the device location. The method also includes displaying a representation of the content item as part of a computer-generated reality scene in which the representation of the content item is positioned relative to the physical environment according to the content position for the content item from the display location information for the content item.

Abstract: A method includes determining a device location of an electronic device, and obtaining a content item to be output for display by the electronic device based on the device location, wherein the content item comprises coarse content location information and fine content location information. The method also includes determining an anchor in a physical environment based on the content item, determining a content position and a content orientation for the content item relative to the anchor based on the fine content location information, and displaying a representation of the content item using the electronic device using the content position and the content orientation.

Abstract: A method includes determining a device location of an electronic device, and obtaining a content item to be output for display by the electronic device based on the device location, wherein the content item comprises coarse content location information and fine content location information. The method also includes determining an anchor in a physical environment based on the content item, determining a content position and a content orientation for the content item relative to the anchor based on the fine content location information, and displaying a representation of the content item using the electronic device using the content position and the content orientation.