Abstract: A conversation augmentation system can automatically augment a conversation with content items based on natural language from the conversation. The conversation augmentation system can select content items to add to the conversation based on determined user “intents” generated using machine learning models. The conversation augmentation system can generate intents for natural language from various sources, such as video chats, audio conversations, textual conversations, virtual reality environments, etc. The conversation augmentation system can identify constraints for mapping the intents to content items or context signals for selecting appropriate content items. In various implementations, the conversation augmentation system can add selected content items to a storyline the conversation describes or can augment a platform in which an unstructured conversation is occurring.

Abstract: A multimedia query system is described that includes a multimedia capture system configured to capture raw multimedia data comprising at least one of raw video data or raw audio data, a metadata engine configured to extract one or more anchor points of metadata from the raw multimedia data and to store the one or more anchor points of metadata, wherein the anchor points of metadata includes references to respective portions of the raw multimedia data. The multimedia query system further includes a storage engine configured to store the raw multimedia data, a recall engine configured to receive a query and to apply the query to the one or more anchor points of metadata to identify one or more raw multimedia data candidates from the portions of the raw multimedia data, and a query engine configured to generate a response to the query based on the one or more raw multimedia data candidates.

Abstract: A multimedia query system is described that includes a multimedia capture system configured to capture raw multimedia data comprising at least one of raw video data or raw audio data, a metadata engine configured to extract one or more anchor points of metadata from the raw multimedia data and to store the one or more anchor points of metadata, wherein the anchor points of metadata includes references to respective portions of the raw multimedia data. The multimedia query system further includes a storage engine configured to store the raw multimedia data, a recall engine configured to receive a query and to apply the query to the one or more anchor points of metadata to identify one or more raw multimedia data candidates from the portions of the raw multimedia data, and a query engine configured to generate a response to the query based on the one or more raw multimedia data candidates.

Abstract: Various client devices include displays and one or more image capture devices configured to capture video data. Different users of an online system are associated with client devices that exchange information captured by their respective image capture devices. A user of a client device may provide a message to an additional user of an additional client device for asynchronous presentation to the additional user. The message includes information identifying the additional user and one or more privacy settings. When the additional client device identifies the user via captured video data, the online system determines whether the message has previously been presented to the additional user. Subject to the privacy settings included in the message, the additional client device presents the message to the additional user after detecting the additional user and determining the message has not previously been presented to the additional user.

Abstract: In some implementations, the disclosed systems and methods can create a customized pet avatar by applying artificial intelligence to photographs and videos of a real-world pet. In some implementations, the disclosed systems and methods can geospatially map user-generated content within a VR environment. In some implementations, the disclosed systems and methods can receive user-generated content (e.g., images, videos, text, etc.) about a particular destination, such as a business listing, restaurant, or other location of interest. In some implementations, the disclosed systems and methods can build a selected virtual object of a plurality of virtual objects in an artificial reality world.

Abstract: This disclosure describes transcribing speech using audio, image, and other data. A system is described that includes an audio capture system configured to capture audio data associated with a plurality of speakers, an image capture system configured to capture images of one or more of the plurality of speakers, and a speech processing engine. The speech processing engine may be configured to recognize a plurality of speech segments in the audio data, identify, for each speech segment of the plurality of speech segments and based on the images, a speaker associated with the speech segment, transcribe each of the plurality of speech segments to produce a transcription of the plurality of speech segments including, for each speech segment in the plurality of speech segments, an indication of the speaker associated with the speech segment, and analyze the transcription to produce additional data derived from the transcription.

Abstract: This disclosure describes a computing system that automatically detects users in visual proximity and adds the users to a private collaboration space enabling the users to share digital content. In one example, the computing system includes a video processing engine configured to detect, from first image data representative of a first physical environment that includes a second user, the second user, wherein the first image data is captured by an image capture system of a head-mounted display (HMD) worn by a first user. The computing system also includes a collaboration application configured to add, in response to detection of the second user, the second user to a set of users associated with a private collaboration space in which the set of users access shared digital content, wherein the set of users includes the first user.

Abstract: Aspects of the present disclosure are directed to an artificial intelligence (“AI”) application running in conjunction with an artificial reality (“XR”) space. The AI Builder responds to user commands, verbal or gestural, to build or edit spaces or objects in space. If the requested object is of a type recognized by the AI Builder, then the AI Builder builds the object from one or more stored templates. The new object's location is determined by the objects that already exist in the user's XR environment and on commands or gestures from the user. If the AI Builder does not recognize the requested object, the user can show an image to the AI Builder, and the AI builds a 3D object in the XR space according to that image. To ease collaboration among users, the AI Builder may present its user interface as a non-player character within the XR world.

Abstract: Aspects of the present disclosure are directed to creating a skybox for an artificial reality (“XR”) world from a two-dimensional (“2D”) image. The 2D image is scanned and split into at least two portions. The portions are mapped onto the interior of a virtual enclosed 3D shape, for example, a virtual cube. A generative adversarial network (GAN) interpolates from the information in the areas mapped from the portions to fill in at least some unmapped areas of the interior of the 3D shape. The 3D shape can be placed in a user's XR world to become the skybox surrounding that world.

Abstract: Various client devices include displays and one or more image capture devices configured to capture video data. Different users of an online system may authorize client devices to exchange information captured by their respective image capture devices. Additionally, a client device modifies captured video data based on users identified in the video data. For example, the client device changes parameters of the image capture device to more prominently display a user identified in the video data and may further change parameters of the image capture device based on gestures or movement of the user identified in the video data. The client device may apply multiple models to captured video data to modify the captured video data or subsequent capturing of video data by the image capture device.

Abstract: A conversation augmentation system can automatically augment a conversation with content items based on natural language from the conversation. The conversation augmentation system can select content items to add to the conversation based on determined user “intents” generated using machine learning models. The conversation augmentation system can generate intents for natural language from various sources, such as video chats, audio conversations, textual conversations, virtual reality environments, etc. The conversation augmentation system can identify constraints for mapping the intents to content items or context signals for selecting appropriate content items. In various implementations, the conversation augmentation system can add selected content items to a storyline the conversation describes or can augment a platform in which an unstructured conversation is occurring.

Abstract: Techniques for enhancing group communication on a device are described. A method may include presenting a message in a message portion of a user interface (UI) for a group communication application executing on a first device, where the group communication application communicates messages among multiple devices. The method may further include monitoring an input component for the first device and detecting a first control directive from the input component. The first control directive may select a message displayed in the message portion of the user interface. The method may further include performing a pin operation on the selected message; and presenting the pinned message in a banner portion of the UL Other embodiments are described and claimed.

Abstract: Techniques for enhancing group communication on a device are described. A method may include presenting a message in a message portion of a user interface (UI) for a group communication application executing on a first device, where the group communication application communicates messages among multiple devices. The method may further include monitoring an input component for the first device and detecting a first control directive from the input component. The first control directive may select a message displayed in the message portion of the user interface. The method may further include performing a pin operation on the selected message; and presenting the pinned message in a banner portion of the UL Other embodiments are described and claimed.

Abstract: Various client devices include displays and one or more image capture devices configured to capture video data. Different users of an online system may authorize client devices to exchange information captured by their respective image capture devices. Additionally, a client device modifies captured video data based on users identified in the video data. For example, the client device changes parameters of the image capture device to more prominently display a user identified in the video data and may further change parameters of the image capture device based on gestures or movement of the user identified in the video data. The client device may apply multiple models to captured video data to modify the captured video data or subsequent capturing of video data by the image capture device.

Abstract: A conversation augmentation system can automatically augment a conversation with content items based on natural language from the conversation. The conversation augmentation system can select content items to add to the conversation based on determined user “intents” generated using machine learning models. The conversation augmentation system can generate intents for natural language from various sources, such as video chats, audio conversations, textual conversations, virtual reality environments, etc. The conversation augmentation system can identify constraints for mapping the intents to content items or context signals for selecting appropriate content items. In various implementations, the conversation augmentation system can add selected content items to a storyline the conversation describes or can augment a platform in which an unstructured conversation is occurring.

Abstract: The present embodiments relate to automated memory creation and retrieval from moment content items. In some implementations, the automated memory creation and retrieval system can obtain moment content items from user-designated sources with a single user perspective or multiple user perspectives. The moment content items can be assigned tags and arranged in chronological order. The arranged moment content items can be clustered into memory content items based on clustering conditions. Once memory content items are created, they can be arranged into a memory hierarchy made up of connected nodes. In some implementations, the memory content items are also linked together based on similarity in various dimensions in a memory graph. The automated memory creation and retrieval system can receive search criteria for memories from a user interface and provide the user with memories from matched nodes in the memory hierarchy or linked memories in the memory graph.

Abstract: The present embodiments relate to automated memory creation and retrieval from moment content items. In some implementations, the automated memory creation and retrieval system can obtain moment content items from user-designated sources with a single user perspective or multiple user perspectives. The moment content items can be assigned tags and arranged in chronological order. The arranged moment content items can be clustered into memory content items based on clustering conditions. Once memory content items are created, they can be arranged into a memory hierarchy made up of connected nodes. In some implementations, the memory content items are also linked together based on similarity in various dimensions in a memory graph. The automated memory creation and retrieval system can receive search criteria for memories from a user interface and provide the user with memories from matched nodes in the memory hierarchy or linked memories in the memory graph.

Abstract: In one embodiment, a method includes detecting objects in an image. The method includes accessing a mask for each object. The method includes receiving an input in relation to the image. The input corresponds to an input region and an input type. The method includes identifying a region of the image corresponding to the input region of the input. The identified region of the image includes one or more of the masks. The method includes providing feedback regarding the one or objects in the identified region of the image based on the input type.

Abstract: Various client devices include displays and one or more image capture devices configured to capture video data. Different users of an online system are associated with client devices that exchange information captured by their respective image capture devices. A receiving client device includes information identifying one or more users of an online system authorized to exchange video data with the receiving client device when the receiving client device determines a receiving user is within a field of view of the receiving client device. Video data is exchanged between the receiving client device and a client device associated with a user authorized to exchange video data without the receiving user providing an input to the receiving client device to accept the video data exchange.

Abstract: Various client devices include displays and one or more image capture devices configured to capture video data. Different users of an online system are associated with client devices that exchange information captured by their respective image capture devices. When exchanging information, presentation of content to users associated with different client device may be initially synchronized across the client devices. To synchronize content presentation, a client device initiating presentation of the content transmits a request identifying the content and an initial time to other client devices. The initial time is greater than a maximum return time or latency in a network coupling the client devices and the online system from a time when the request is transmitted. A client device determined to be out of synchronization with one or more other client devices receives a command to modify a rate at which the content is presented to reestablish synchronization.