Abstract: One embodiment of a computer-implemented method for analyzing human motion data includes receiving a set of motion data that indicates one or more movements of a first person within a real-world environment; generating a virtual avatar corresponding to the first person based on the set of motion data; determining a position of the virtual avatar within an extended reality (ER) scene based on the one or more movements; and displaying the virtual avatar in the ER scene according to the determined position.

Abstract: A command map GUI that illustrates command usage patterns of one or more entities. A server receives and stores command usage data from a plurality of users/clients, each user/client executing a software application having a set of commands. A current user/client displays, via a GUI, graphical representations of the command usage data received from the server. The server receives new command usage data from a user/client included in the one or more entities. In response, the server automatically updates command usage data for the one or more entities, pushes the updated command usage data to the current user/client, and causes the GUI to display graphical representations of the updated command usage data. Thus, the command usage data of the one or more entities is automatically updated and displayed in the GUI in real-time or near real-time as new commands are continually invoked by users in the one or more entities.

Abstract: One embodiment of the present invention sets forth a technique for providing awareness of privacy-related activities. The technique includes determining a privacy level associated with a user of an extended reality environment. The technique also includes presenting, using an internal display of a headset, one or more internal indicators identifying a location of a bystander located in a real-world environment, wherein a level of detail of each internal indicator is based on the privacy level associated with the user. The technique further includes presenting, using an external display, one or more external indicators that include a monitoring indicator representing being captured by the headset and presented to the user via the headset, and further include a user activity indicator representing one or more activities of the user, wherein a level of detail of the user activity indicator is based on the privacy level associated with the user.

Abstract: A workstation enables operation of a 2D input device with a 3D interface. A cursor position engine determines the 3D position of a cursor controlled by the 2D input device as the cursor moves within a 3D scene displayed on a 3D display. The cursor position engine determines the 3D position of the cursor for a current frame of the 3D scene based on a current user viewpoint, a current mouse movement, a CD gain value, a Voronoi diagram, and an interpolation algorithm, such as the Laplacian algorithm. A CD gain engine computes CD gain optimized for the 2D input device operating with the 3D interface. The CD gain engine determines the CD gain based on specifications for the 2D input device and the 3D display. The techniques performed by the cursor position engine and the techniques performed by the CD gain engine can be performed separately or in conjunction.

Abstract: Methods, systems, and apparatus, including medium-encoded computer program products, for searching through spatial recordings by constructing queries. One of the methods include constructing a query based on input received at a three-dimensional (3D) space displayed at a visual programming interface, wherein the input includes positioning of objects at 3D positions within the 3D space over time, wherein the input defines at least one spatial orientation between at least two objects from the objects in the 3D space; executing the query to search a database of 3D recordings to find at least one segment from at least one 3D recording that includes the at least two objects and matches the spatial orientation between the at least two objects, as defined in the input, at a specific point in time; and presenting the at least one segment from the at least one 3D recording via the visual programming interface.

Abstract: One embodiment of the present invention sets forth a technique for designing and generating a smart object. The technique includes receiving a first input indicating a smart object behavior of a smart object that includes a smart device embedded in a three-dimensional (3D) object; in response to the input, generating computer instructions for the smart device, wherein the computer instructions, when executed by the smart device, cause the smart object to implement the smart object behavior; and transmitting the computer instructions to the smart device.

Abstract: A workstation enables operation of a 2D input device with a 3D interface. A cursor position engine determines the 3D position of a cursor controlled by the 2D input device as the cursor moves within a 3D scene displayed on a 3D display. The cursor position engine determines the 3D position of the cursor for a current frame of the 3D scene based on a current user viewpoint, a current mouse movement, a CD gain value, a Voronoi diagram, and an interpolation algorithm, such as the Laplacian algorithm. A CD gain engine computes CD gain optimized for the 2D input device operating with the 3D interface. The CD gain engine determines the CD gain based on specifications for the 2D input device and the 3D display. The techniques performed by the cursor position engine and the techniques performed by the CD gain engine can be performed separately or in conjunction.

Abstract: In various embodiments a computer-implemented method for providing sustainability insights to a user designing an object. The method includes determining a first value of a sustainability metric associated with a design of an object, displaying, via a graphical user interface (GUI), a visual indication of the first value of the sustainability metric, and detecting a change to the design of the object. The method further includes, in response to detecting the change to the design of the object, determining a second value of the sustainability metric and displaying, via the GUI, a visual indication of the second value of the sustainability metric.

Abstract: In various embodiments a computer-implemented method for providing sustainability insights to a user designing an object. The method includes determining a value of a sustainability metric associated with a design of an object; determining a target value for the sustainability metric; determining an amount of progress made towards achieving the target value for the sustainability metric based on the value of the sustainability metric and the target value for the sustainability metric; and displaying, via a graphical user interface (GUI), a visual indication of the amount of progress made towards achieving the target value for the sustainability metric.

Abstract: One embodiment of the invention disclosed herein provides a system that includes a mirror apparatus with a first surface to which a half-silvered mirror film is applied, where the mirror apparatus transmits a transmitted image from a second surface to the first surface. The system further includes a servo-controlled dimmer that adjusts a level of ambient light associated with the mirror apparatus. The system further includes a motion sensing device that tracks positions of a plurality of points associated with an object; wherein the object is situated on the half-silvered mirror film side of the mirror apparatus. The system further includes a computing device including a memory that stores instructions that, when executed by a processor included in the computing device, cause the processor to control the servo-controlled dimmer to adjust the ambient light such that both the transmitted image and a reflected image is visible on the first surface.

Abstract: Methods, systems, and apparatus, including medium-encoded computer program products, for providing editable keyframe-based animation data for applying to a character to animate motion of the character in three-dimensional space. Three-dimensional motion data is constructed from two-dimensional videos. The three-dimensional motion data represents movement of people in the two-dimensional videos and includes, for each person, a root of a three-dimensional skeleton of the person. The three-dimensional skeleton comprises multiple three-dimensional poses of the person during at least a portion of frames of a video from the two-dimensional videos. The three-dimensional motion data is converted into editable keyframe-based animation data in three-dimensional space and provided to animate motion.

Abstract: Methods, systems, and apparatus, including medium-encoded computer program products, for collaboration and view sharing between users when performing editing operations over a shared document. A first portion of a shared document is displayed to a first user in a user interface of a first instance of a collaboration application of a first user. The displayed first portion comprises a first location of the first user within the shared document. In the user interface, an indication specifying a relative locational direction from the first location towards a second location of a second user within the shared document is provided. A second portion of the shared document is being displayed to the second user through a second instance of the collaboration application during a conference call between a set of users, where the displayed second portion includes the second location of the second user within the shared document.

Abstract: A hybrid workstation enables a virtual reality (VR) interface, a traditional (TD) interface, and transitions between the interfaces. The VR interface comprises three-dimensional (3D)-based software and hardware components. The TD interface comprises two-dimensional (2D)-based software and hardware components. The state of the hybrid workstation is defined by three parameters comprising interface (VR interface or TD interface), position (seated or standing), and movement (stationary or room-scale). The hybrid workstation detects a transition from a current state to a next state upon determining that any of the three parameters have changed. The hybrid workstation then determines a transition response based on the particular transition that is detected. The transition response comprises a set of operations that are performed on the VR interface and/or the TD interface that mitigate the disruption and inefficiency caused when the particular transition occurs.

Abstract: Techniques disclosed herein combine computer vision with eye tracking by identifying, via computer vision, safety hazards in a video captured by a camera mounted on a pair of safety glasses, and generating an alert if a user wearing the safety glasses has not (recently) noticed the hazard. Whether the user has noticed the hazard is determined based on eye tracking information extracted from a video captured by another camera that is mounted on the safety glasses and points toward the user. As a result, safety hazards may be automatically detected and reported to the user. In addition, only those hazards that have not been (recently) noticed by the user cause an alert to be generated, so the user is not distracted with unnecessary notifications.

Abstract: A computer-implemented method for generating a virtual collection of digital materials includes: generating a virtual three-dimensional (3D) design workspace; and, in response to a first operation that is associated with a first media file: automatically generating a virtual 3D representation of the first media file; and automatically incorporating the virtual 3D representation of the first media file into the virtual 3D design workspace.

Abstract: Approaches for generating virtual representations of smart objects in a 3D visual programming interface. The interface displays a 3D virtual environment containing virtual objects that represents a real-world environment containing smart objects. The 3D virtual environment displays virtual objects in a manner that is spatially accurate relative to the physical objects in the real-world environment. For each virtual object representing a physical object, a logic node (port node) is displayed, the port node representing the set of functions associated with the physical object. The interface enables users to create, delete, or modify different types of logic nodes (representing functions) and create, delete, or modify links (representing data connections) between logic nodes within the 3D virtual environment. The authoring of the logic nodes and links produces an executable program.

Abstract: An automated robot design pipeline facilitates the overall process of designing robots that perform various desired behaviors. The disclosed pipeline includes four stages. In the first stage, a generative engine samples a design space to generate a large number of robot designs. In the second stage, a metric engine generates behavioral metrics indicating a degree to which each robot design performs the desired behaviors. In the third stage, a mapping engine generates a behavior predictor that can predict the behavioral metrics for any given robot design. In the fourth stage, a design engine generates a graphical user interface (GUI) that guides the user in performing behavior-driven design of a robot. One advantage of the disclosed approach is that the user need not have specialized skills in either graphic design or programming to generate designs for robots that perform specific behaviors or express various emotions.

Abstract: One embodiment of a computer-implemented method for analyzing human motion data includes receiving a set of motion data that indicates one or more movements of a first person within a real-world environment; generating a virtual avatar corresponding to the first person based on the set of motion data; determining a position of the virtual avatar within an extended reality (ER) scene based on the one or more movements; and displaying the virtual avatar in the ER scene according to the determined position.

Abstract: Methods, systems, and apparatus, including medium-encoded computer program products, for providing editable keyframe-based animation data for applying to a character to animate motion of the character in three-dimensional space. Three-dimensional motion data is constructed from two-dimensional videos. The three-dimensional motion data represents movement of people in the two-dimensional videos and includes, for each person, a root of a three-dimensional skeleton of the person. The three-dimensional skeleton comprises multiple three-dimensional poses of the person during at least a portion of frames of a video from the two-dimensional videos. The three-dimensional motion data is converted into editable keyframe-based animation data in three-dimensional space and provided to animate motion.

Abstract: In various embodiments, a mentor application automatically obtains assistance with software applications. The mentor application generates a computer-generated help request associated with a first user of a software application. Based on the computer-generated help request and a set of user contexts associated with a set of users, the mentor application computes match scores. Each match score predicts how suitable a particular user is for servicing the computer-generated help request. Based on the match scores, the mentor application transmits at least one help request notification to at least one user included in the set of users to determine a second user to service the computer-generated help request. The mentor application then establishes a computer connection between the first user and the second user through which an interactive help session between the first user and the second user is held.