Abstract: Systems and methods for obtaining user requests and assisting with user tasks based on the user requests at a head-wearable device and a wrist-wearable device are disclosed. The method includes, while a user is wearing a display-less head-wearable device including a camera and a wrist-wearable device including a display, in response to obtaining a user request, determining, based on the user request, a user task, causing the display of the wrist-wearable device to present a user interface associated with the user task, causing the camera of the display-less head-wearable device to capture image data, and in accordance with a determination, based on the image data, that the user has completed a portion of the user task, causing the user interface presented at the display of the wrist-wearable device to be updated based on completion of the portion of the user task.

Abstract: Systems and methods for transitioning between power modes of a wearable device are disclosed. An example method includes, while a user is wearing the head-wearable device, obtaining first real-world data of the user's surroundings captured while the head-wearable device is operating in a low-power mode. The method includes, in accordance with a determination, based on the first real-world data, that an AI agent invocation trigger is satisfied, obtaining second real-world data of the user's surroundings captured while the head-wearable device is operating in a high-power mode. The method includes, in accordance with a determination, based on the second real-world data, that AI assistance criteria are satisfied causing the AI agent to generate assistive operations, causing the head-wearable device to present the assistive operations, and in response to user selection of an assistive operation of the assistive operations, causing the AI agent to perform the assistive operation.

Abstract: Systems and methods of capturing and collecting image data and generating composite images are disclosed. An example method includes, while the wearable device is worn by a user, determining whether capture criteria are satisfied. The method includes, in accordance with a determination that the capture criteria are satisfied, capturing, via the imaging device, first image data. The method also includes, in accordance with a determination that the wearable device is participating in a group activity with at least one other electronic device associated with another user, obtaining second image data from the at least one other electronic device. The method further includes generating, based on the first image data and the second image data, a composite image including portions of the first image data and the second image data.

Abstract: Systems and methods for generating and distributing notes using artificially intelligence are disclosed. An example method includes obtaining an indication that a user wearing a head-wearable device is participating in an information sharing event and, in response to the indication, causing an imaging device to capture image data and a microphone to capture audio data. The method includes providing the image data and the audio data to an artificial intelligence (AI) agent, receiving, based on the image data and the audio data provided to the AI agent, AI-generated notes including note items. The note items include an audio recording of information shared during the information sharing event, a transcription of information shared during the information sharing event, a summary of the information sharing event, representations of the image data captured during the information sharing event, and/or action items determined based on the information shared during the information sharing event.

Abstract: Aspects of the present disclosure are directed to managing mixed reality and virtual reality operating modes using an artificial reality system. An artificial reality system can be capable of operating in a virtual reality mode, where the environment displayed to a user is computer generated, or mixed reality mode, where the environment displayed to a user represents the user's real-world environment but includes one or more virtual objects that are computer generated. Implementations of a mode manager can manage these operating modes of the artificial reality system, such as managing the operating mode an XR system boots into on startup, managing transitions between these operating modes, and managing the display of virtual objects that persist in both operating modes.

Abstract: Systems and methods of traversing webpages using an artificial intelligence (AI) agent are disclosed. An example method includes, in response to a user request, received via a wearable device worn by a user, to perform a web-based action, invoking the AI agent. The method includes, causing the AI agent to traverse webpages based on the user request. The AI agent obtains, for each webpage of the webpages, information for performing the web-based action. The method includes generating, by the AI agent, an AI response based on the information, and presenting, at the wearable device, the AI response. The AI response includes recommendations for performing the web-based action. The method further includes presenting, at the wearable device, the AI response; and, in response to user selection, via the wearable device, of a recommendation included in the AI response, causing the AI agent to perform the web-based action based on the recommendation.

Abstract: Systems and methods of providing personalized sports intelligence using an artificial intelligence are disclosed. An example method includes, while a user wearing a head-wearable device is performing a physical activity, obtaining sensor data sensed from the user via one or more sensors, the sensor data including at least biometric data and physical activity data. The method includes, in response to an indication that the user is performing the physical activity, initiating an artificial intelligence (AI) agent. The method also includes determining, based on the sensor data, whether one or more activity-based thresholds are satisfied and, in accordance with a determination that an activity-based threshold is satisfied, generating, by the AI agent, a coaching message to assist the user in performance of the physical activity. The method further includes causing presentation of the coaching message via the head-wearable device.

Abstract: Systems and method of providing personalized physical activity notifications are disclosed. An example method is performed by a system including one or more processors communicatively coupled with a head-wearable device and one or more sensors. The method includes, while a user wearing the head-wearable device is performing a physical activity, obtaining sensor data sensed from the user via the one or more sensors. The sensor data includes at least biometric data and physical activity data. The method includes, in response to a user input querying the physical activity, initiating an artificial intelligence (AI) agent, and generating, by the AI agent, an AI message responsive to the user input. The AI message is based on one or more of the sensor data and the physical activity data. The method also includes causing presentation of the AI message via the head-wearable device.

Abstract: Systems and method of generating orchestrated guidance based on an activity of a user are disclosed. An example method for generating orchestrated guidance based on an activity of a user includes in response to an indication received at a wearable device that an artificial intelligence (AI) agent trigger condition is present, providing an AI agent sensor data obtained by the wearable device. The method includes determining, by the AI agent, a context-based activity based on the sensor data obtained by the wearable device and generating, by the AI agent, orchestrated guidance based on the context-based activity. The orchestrated guidance includes a recommended action for performing the context-based activity. The method also includes presenting the orchestrated guidance at the wearable device.

Abstract: Aspects of the present disclosure can provide portals to virtual reality (VR) experiences from within an artificial reality (XR) entry space, such as mixed reality (MR) or augmented reality (AR). Some implementations can allow a user to move between pre-established physical spaces that are mapped to certain activities and tagged content. Some implementations can tag the content to a particular real-world location based on user input, a system suggestion, or the system simply picking up where the user left off in an AR or MR experience in that location. In some implementations, the content can launch a corresponding VR experience. For example, a virtual object can appear overlaid on a user's real-world room. By grabbing the virtual object, some implementations can automatically launch a VR experience. In another example, some implementations can automatically display a virtual portal to a VR co-working experience when the user sits at her desk.

Abstract: In one embodiment, a method includes rendering, for a virtual reality (VR) display device, a first output image of a VR environment. The VR environment includes a personal user interface (UI) associated with a first user. The personal UI has a first form factor and a first pose with respect to the first user. The personal UI is a virtual object. The method includes detecting a change in context of the first user with respect to a real-world object. The method includes determining whether to adapt the personal UI based on a detected change in the context of the first user with respect to the real-world object. The method includes rendering, for the VR display device, a second output image of the VR environment. The personal UI is adapted to have a second form factor and a second pose with respect to the first user responsive to determining to adapt the personal UI. The second form factor and the second pose of the personal UI is based on the real-world object.

Abstract: In one embodiment, a method includes rendering, for one or more displays of a VR display device, a first output image of a VR environment. The VR environment includes a personal UI associated with a first user. The personal UI has a first form factor and a first pose with respect to the first user, is a virtual object comprising one or more 2D virtual displays, and is operable to execute a plurality of applications. The method includes determining whether to adapt the personal UI based on a detected change in a context of the first user with respect to the VR environment. The method includes rendering a second output image of the VR environment, where the personal UI is adapted to have a second form factor and a second pose with respect to the first user responsive to determining to adapt the personal UI based on the detected change in the context of the first user.

Abstract: In one embodiment, a method includes displaying, for one or more displays of a virtual VR device, a first output image comprising a passthrough view of a real-world environment. The method includes identifying, using one or more images captured by one or more cameras of the VR display device, a real-world object in the real-world environment. The method includes receiving a user input indicating a first dimension corresponding to the real-world object. The method includes automatically determining, based on the first dimension, a second and third dimension corresponding to the real-world object. The method includes rendering, for the one or more displays of the VR display device, a second output image of a VR environment. The VR environment includes a MR object that corresponds to the real-world object. The MR object is defined by the determined first, second, and third dimensions.

Abstract: In one embodiment, a method includes displaying, for one or more displays of a virtual VR device, a first output image comprising a passthrough view of a real-world environment. The method includes identifying, using one or more images captured by one or more cameras of the VR display device, a real-world object in the real-world environment. The method includes receiving a user input indicating a first dimension corresponding to the real-world object. The method includes automatically determining, based on the first dimension, a second and third dimension corresponding to the real-world object. The method includes rendering, for the one or more displays of the VR display device, a second output image of a VR environment. The VR environment includes a MR object that corresponds to the real-world object. The MR object is defined by the determined first, second, and third dimensions.

Abstract: In one embodiment, a method includes rendering, for one or more displays of a VR display device, a first output image of a VR environment. The VR environment includes a personal UI associated with a first user. The personal UI has a first form factor and a first pose with respect to the first user, is a virtual object comprising one or more 2D virtual displays, and is operable to execute a plurality of applications. The method includes determining whether to adapt the personal UI based on a detected change in a context of the first user with respect to the VR environment. The method includes rendering a second output image of the VR environment, where the personal UI is adapted to have a second form factor and a second pose with respect to the first user responsive to determining to adapt the personal UI based on the detected change in the context of the first user.

Abstract: The technology described in this document is embodied in a method that includes deriving a metric associated with a state of a subject, the state of the subject being one or more members selected from the group consisting of health, sleep, fitness, and stress. Deriving the metric is based on data in a first dataset that represents time-varying information about at least one pulse pressure wave propagating through blood in the subject acquired at a location of the subject.