Abstract: The present disclosure pertains to techniques for reducing inaccuracies in context aware policies (CAP) with real-time feedforward validation in extended reality environments. In a particular aspect, a extended reality system is configured to author, by a user using the head-mounted device, a rule or policy or context aware policy (CAP) including one or more target actions and triggering context instances, capture, using one or more sensors, context history records for the user, render, by the head-mounted device, one or more validation scenes based on the rule or policy or CAP and the context history records, validate, by the user using the head-mounted device, the one or more validation scenes within an extended reality environment, and update, by the user using the head-mounted device, the rule or policy or CAP based on the validation.

Abstract: Features described herein generally relate to learning and recognizing object-centered routines. Particularly, object-centered routines can be learned and recognized by collecting data corresponding to a user. The data can include information representing interactions by the user with respect to objects in an environment. The routine can be learned by presenting a visual graph to the user. The user can define nodes associating an interaction with an object, specify a relationship between nodes, and arrange the nodes into segments. The visual graph can be stored, and a routine can be recognized based on the visual graph.

Abstract: An Adaptive Tangible User Interface (ATUI) for use in an extended reality environment in which tangible interfaces are composed in real time based on identified affordances of existing objects in the physical environment and the input tasks of a user. An extended reality system can be provided with instructions executable by one or more processors to perform processing including: generating a representation of the real-world environment within a field of view of the user; identifying physical objects within the real-world environment; generating a set of possible performable gestures afforded by available object affordance factors; determining potential input tasks; composing performable gestures for the potential input tasks; and selecting a physical object for use as an adaptive tangible user interface. Techniques for designing a virtual user interface, overlaying the virtual user interface on a selected ATUI physical object, and maintaining alignment of the virtual user interface, are also provided.

Abstract: The present disclosure relates to defining and modifying behavior in an extended reality environment. The systems and methods include capturing, using the one or more audio sensors, a natural language explanation of a rule or policy from the user, extracting features from the natural language explanation of the rule or policy, wherein the features include one or more conditions, one or more actions, and connections between the one or more events, conditions, and actions, predicting a control structure comprised of one or more conditional statements based on the extracted features and model parameters learned from historical rules or policies, and generating the rule or policy based on the control structure, wherein the rule or policy comprises the one or more conditional statements for executing the one or more actions based on evaluation of the one or more conditions.

Abstract: Features described herein generally relate to generating and modifying policies with an artificial intelligence (AI) platform based on user activities. Particularly, data comprising characteristics of activities performed by a user in real-world and virtual environments is collected, a control structure is predicted based on the data and model parameters learned from historical policies, and a new policy is generated or a pre-existing policy is modified based on the control structure.

Abstract: Features described herein generally relate to predicting policies with an artificial intelligence (AI) platform based on shared or similar interactions. Particularly, data that includes information about users and a corpus of policies is collected, embeddings are generated for the collected data, and policies are predicted based on the embeddings. The policies can be predicted based on content-based filtering, collaborative filtering, and/or game theory approaches.

Abstract: The present disclosure relates to refining context aware policies in extended reality systems. In one aspect, an extended reality system is provided that performs operations including: accessing data collected from user interactions while using a context aware policy in an extended reality environment, determining a support set and confidence score for the context aware policy based on the data, generating replacement policies for the context aware policy, determining a support set and confidence score for each of the replacement policies based on the data, identifying a replacement policy from the replacement policies as a replacement for the context aware policy based on the support sets and confidence scores, and updating one or more conditions or an action defined by the context aware policy with a modified version of the one or more conditions or the action defined by the replacement policy to generate an updated context aware policy.

Abstract: The present disclosure relates to systems and methods for defining and modifying behavior in an extended reality environment. The systems and methods include receiving a request to build or modify a rule or policy for a given user or activity, obtaining a template based on the user or the activity, displaying the template and tools for editing the template, receiving a modification to the template via the user interacting with the tools of the user interface, and modifying the template based on the modification to generate the rule or policy. The modification includes changing the initial value for the actions or the conditions of the template and saving the template as a separate instance representative of the rule or policy, and the rule or policy includes one or more conditional statements for executing one or more of the actions based on evaluation of one or more of the conditions.

Abstract: Techniques for analyzing contextual clues from an extended reality environment and, based on the analysis of the contextual clues, intuitively superimposing and integrating customized digital information into the artificial reality environment via a virtual assistant to recommend and lead the user into suggested action. In one particular aspect, a computer-implements method is provided that includes obtaining input data from a user, generating a graph of objects, attributes, and relationships between objects extracted from the input data, determining one or more interactions to be presented, initiated, or executed based on the graph and a profile associated with the user, determining virtual content data to be used for rendering virtual content based on the one or more interactions, and rendering the virtual content in an extended reality environment displayed to the user based on the virtual content data. The virtual content is used to present, initiate, or execute the one or more interactions for the user.