Abstract: In some implementations, a method includes obtaining environmental data corresponding to an environment. The method includes determining that the environmental data corresponding to the environment includes environmental data that corresponds to a first portion of an object that represents a sub-portion of the object and not an entirety of the object. The method includes generating a plurality of candidate point clouds for a second portion of the object based on the environmental data corresponding to the first portion of the object. The plurality of candidate point clouds are associated with corresponding confidence scores. The method includes synthesizing a model of the environment that includes a point cloud representing the first portion of the object, at least a subset of the plurality of candidate point clouds for the second portion of the object and the corresponding confidence scores associated with the subset of the plurality of candidate point clouds.

Abstract: Various implementations disclosed herein include devices, systems, and methods for selecting a point-of-view (POV) for displaying an environment. In some implementations, a device includes a display, one or more processors, and a non-transitory memory. In some implementations, a method includes displaying a first view of a target located in a graphical environment, wherein the first view is associated with a first rig. In some implementations, the method includes detecting a change in the graphical environment. In some implementations, the method includes, in response to detecting the change in the graphical environment, switching from the first rig to a second rig that provides a second view of the target that is different from the first view.

Abstract: A method includes obtaining, at a motion planner, a request to provide a motion plan for an agent that is associated with a motion controller that controls movement of a representation of the agent in an environment. The method includes determining, by the motion planner, an input characteristic of the motion controller that controls movement of the representation of the agent. The method includes generating, by the motion planner, the motion plan for the agent based on the input characteristic of the motion controller. The method includes triggering the motion controller to manipulate the representation of the agent within the environment in accordance with the motion plan generated by the motion planner.

Abstract: In some implementations, the method includes: obtaining a plurality of animations; determining initial and end motion states for each of the plurality of animations; generating an animation graph including nodes for each of the plurality of animations by connecting, with a directional edge, a first node with an end motion state to a second node with an initial motion state that matches the end motion state of the first node; generating a transitional animation that is not included among the plurality of animations from an initial reference motion state to a target motion state that corresponds to a path that traverses the animation graph from a third node associated with the initial reference motion state to a fourth node associated with the target motion state; and updating the animation graph by removing one or more nodes from the animation graph based at least in part on the transitional animation.

Abstract: A device obtains a request to display a graphical environment. The graphical environment is associated with a set of saliency values corresponding to respective portions of the graphical environment. A POV for displaying the graphical environment is selected based on the set of saliency values. The graphical environment is displayed from the selected POV on the display.

Abstract: In one implementation, a method of presenting a scene content is performed at a device including a display, one or more processors, and non-transitory memory. The method includes generating a definition of a scene based on a model of an environment, wherein the definition includes a constraint that defines a spatial relationship between a first asset and a second asset that corresponds to one or more objects in the environment. The method includes generating, based on application of the definition of the scene to first and second environments, first and second instances of the scene that satisfy the constraint with the first asset in the spatial relationship with a first one and a second one of the one or more objects in the first and second environments, respectively. The method includes presenting, on the display, the first and second instances of the scene.

Abstract: In some implementations, a device includes one or more processors and a non-transitory memory. In some implementations, a method includes displaying an object within an extended reality (XR) environment, the object comprising a plurality of portions including a first portion and a second portion. In some implementations, the method includes acquiring gaze data from a user interacting with the XR environment. In some implementations, the method includes determining, based on the gaze data, a first saliency value for the first portion of the object. In some implementations, the method includes triggering presentation of the first portion of the object in accordance with a first display characteristic that is based on the first saliency value.

Abstract: In one implementation, a method of composing a scene content is performed at a device including a display, one or more processors, and non-transitory memory. The method includes generating a definition of a scene based on textual or speech input and a model of a physical environment, wherein the definition includes a constraint that defines a spatial relationship between a virtual asset and an anchor asset that corresponds to one or more physical objects in the physical environment. The method includes generating, based on the definition of the scene and the model of the physical environment, a first instance of the scene that satisfies the constraint with the virtual asset in the spatial relationship with a first one of the one or more physical objects in the physical environment. The method includes presenting, on the display, the first instance of the scene.

Abstract: In some implementations, a device includes one or more processors and a non-transitory memory. In some implementations, a method includes obtaining an object that includes a plurality of portions including a first portion and a second portion. In some implementations, the method includes determining a first saliency value for the first portion of the object and a second saliency value for the second portion of the object. In some implementations, the method includes triggering presentation of the first portion of the object in accordance with a first display characteristic that is based on the first saliency value and the second portion of the object in accordance with a second display characteristic that is based on the second saliency value.

Abstract: Various implementations disclosed herein include a method performed by an electronic device including one or more processors and a non-transitory memory. The method includes obtaining first environmental data that characterizes a first physical setting that is spatially distinct from a second physical setting. The second physical setting includes the electronic device. The method includes generating, using the first environmental data, a first extended reality (XR) environment corresponding to the first physical setting. The method includes generating a first mapping between a first bounding surface of the second physical setting and a first bounding surface of the first XR environment.

Abstract: In one implementation, a method of displaying content is performed at a device including a display, one or more processors, and non-transitory memory. The method includes scanning a first physical environment to detect a first physical object in the first physical environment and a second physical object in the first physical environment, wherein the first physical object meets at least one first object criterion and the second physical object meets at least one second object criterion. The method includes displaying, in association with the first physical environment, a virtual object moving along a first path from the first physical object to the second physical object.

Abstract: According to various implementations, a method is performed at an electronic device including one or more processors, a non-transitory memory, and a display. The method includes displaying, on the display, a virtual agent that is associated with a first viewing frustum. The first viewing frustum includes a user avatar associated with a user, and the user avatar includes a visual representation of one or more eyes. The method includes, while displaying the virtual agent associated with the first viewing frustum, obtaining eye tracking data that is indicative of eye behavior associated with an eye of the user, updating the visual representation of one or more eyes based on the eye behavior, and directing the virtual agent to perform an action based on the updating and scene information associated with the electronic device.

Abstract: In one implementation, a method for recording an XR environment. The method includes: presenting, via the display device, a graphical environment with one or more virtual agents, wherein the graphical environment corresponds to a composition of extended reality (XR) content, including the one or more virtual agents, and an image stream of a physical environment captured from a first point-of-view (POV) of the physical environment; detecting, via the one or more input devices, a user input selecting a first virtual agent from among the one or more virtual agents; and in response to detecting the user input, recording a plurality of data streams associated with the graphical environment including a first image stream of the graphical environment from the first POV and one or more data streams of the graphical environment from a current POV of the first virtual agent.

Abstract: In one implementation, a method of displaying content is performed at a device including a display, one or more processors, and non-transitory memory. The method includes scanning a first physical environment to detect a first physical object in the first physical environment and a second physical object in the first physical environment, wherein the first physical object meets at least one first object criterion and the second physical object meets at least one second object criterion. The method includes displaying, in association with the first physical environment, a virtual object moving along a first path from the first physical object to the second physical object.

Abstract: Various implementations disclosed herein include devices, systems, and methods for training of an action determining component of a computer character. In some implementations, actions are taken by the character in a 3D environment according to an action determining component of the character, where the character is rewarded or penalized for interactions associated with an object/concept in the 3D environment according to an assigned object/concept reward or penalty. In some implementations, the reward or the penalty assigned to the object/concept is modified, and the character is then rewarded or penalized for interactions associated with the object/concept according to the modified reward or the modified penalty. The action determining component of the character is trained using a reinforcement learning technique that accounts for rewards or penalties obtained by virtual character for interactions associated with the object/concept.

Abstract: In one implementation, a method of achieving an objective using a scene graph includes identifying a set of transition couples of a plurality of transition couples between sets of spatial relationships of the scene graph, wherein each spatial relationship of a respective first set of spatial relationships indicated by each transition couple of the set of transition couples is included in an initial set of spatial relationships or an respective second set of spatial relationships indicated by a previous transition couple and wherein a particular spatial relationship is included in a respective second set of spatial relationships of a last transition couple of the set of transition couples.

Abstract: In one implementation, a method for recording an XR environment. The method includes: presenting, via the display device, a graphical environment with one or more virtual agents, wherein the graphical environment corresponds to a composition of extended reality (XR) content, including the one or more virtual agents, and an image stream of a physical environment captured from a first point-of-view (POV) of the physical environment; detecting, via the one or more input devices, a user input selecting a first virtual agent from among the one or more virtual agents; and in response to detecting the user input, recording a plurality of data streams associated with the graphical environment including a first image stream of the graphical environment from the first POV and one or more data streams of the graphical environment from a current POV of the first virtual agent.

Abstract: According to various implementations, a method is performed at an electronic device including one or more processors, a non-transitory memory, and a display. The method includes displaying, on the display, a virtual agent that is associated with a first viewing frustum. The first viewing frustum includes a user avatar associated with a user, and the user avatar includes a visual representation of one or more eyes. The method includes, while displaying the virtual agent associated with the first viewing frustum, obtaining eye tracking data that is indicative of eye behavior associated with an eye of the user, updating the visual representation of one or more eyes based on the eye behavior, and directing the virtual agent to perform an action based on the updating and scene information associated with the electronic device.

Abstract: In one implementation, a method for recording an XR environment. The method includes: presenting, via the display device, a graphical environment with one or more virtual agents, wherein the graphical environment corresponds to a composition of extended reality (XR) content, including the one or more virtual agents, and an image stream of a physical environment captured from a first point-of-view (POV) of the physical environment; detecting, via the one or more input devices, a user input selecting a first virtual agent from among the one or more virtual agents; and in response to detecting the user input, recording a plurality of data streams associated with the graphical environment including a first image stream of the graphical environment from the first POV and one or more data streams of the graphical environment from a current POV of the first virtual agent.

Abstract: In some implementations, a device includes one or more sensors, one or more processors and a non-transitory memory. In some implementations, a method includes determining that a first portion of a physical environment is associated with a first saliency value and a second portion of the physical environment is associated with a second saliency value that is different from the first saliency value. In some implementations, the method includes obtaining, via the one or more sensors, environmental data corresponding to the physical environment. In some implementations, the method includes generating, based on the environmental data, a model of the physical environment by modeling the first portion with a first set of modeling features that is a function of the first saliency value and modeling the second portion with a second set of modeling features that is a function of the second saliency value.