Abstract: A display of an augmented reality-enabled (AR) device, such as a mobile phone, can be used to transfer a graphical object between a secondary display, such as a computer monitor, that is captured by a camera of the AR device, and AR space, where the object is visible only through the AR interface of the AR device. A graphical object can be selected through the AR interface and, for example, moved around on a canvas of the secondary display by the user of the AR device. When the AR interface is used to move an enabled object near an edge of the canvas or physical boundary of the secondary display, the object as shown on the secondary display can be made to disappear from the secondary display to be replaced by a virtual object shown only on the AR interface in a similar location.

Abstract: In one embodiment, a method includes instructing, at a first time, a camera having a plurality of pixel sensors to capture a first image of an environment comprising an object to determine a first object pose; determining, based on the first object pose, a predicted object pose of the object at a second time; generating pixel-activation instructions based on a buffer region around a projection of a 3D model of the object having the predicted object pose onto a virtual image plane associated with a predicted camera pose, where the size of the buffer region may be dependent on predicted dynamics for the object; instructing, at the second time, the camera to use a subset of the plurality of pixel sensors to capture a second image of the environment according to the pixel-activation instructions, and; determining, based on the second image, a second object pose of the object.

Abstract: A computer generated (CG) hair groom for a virtual character can include strand-based (also referred to as instanced) hair in which many thousands of digital strands represent real human hair strands. Embodiments of systems and methods for transferring CG hair groom data from a first (or source) virtual character to a second (or target) virtual character are provided. Some embodiments can factor in a difference between a hairline of the first virtual character and a hairline of the second virtual character to improve the overall appearance or fit of the hair groom on the second virtual character.

Abstract: A computer-implemented method includes capturing visual data of an environment using an image sensor of an electronic device and non-visual data of the environment using one or more non-image sensors of the electronic device. Feature descriptors of one or more objects in the environment are generated using the visual data of the environment and the non-visual data of the environment. A map of the environment is generated using the feature descriptors of the one or more objects. One or more virtual objects are anchored to at least one of the objects using the map. The visual data, the non-visual data, and the map are combined in a digital multimedia container file. The digital multimedia container file is stored on the electronic device or on another electronic device connected to the electronic device.

Abstract: An environment association system (“EAS”) comprising: a processor and a memory; an object recognition process configured to identify objects within images, the objects including one or more of a vehicle, a vehicle lift, a vehicle repair tool, and an alignment fixture; and an EAS interface configured to communicate with a user device, the user device comprising a camera and a display; wherein the processor is configured to: determine, for at least one object in the set of objects, create a virtual overlay for the image based on the position of the at least one object within the image and a virtual marking associated with the at least one object; and provide the virtual overlay to the user device, wherein the virtual overlay is configured to cause the user device to simultaneously display the image and the virtual overlay via the display. The system provides information about the alignment of lifting points of the object with lifting members.

Abstract: Systems and methods for displaying a virtual character in a mixed reality environment are disclosed. In some embodiments, a view of the virtual character is based on an animation rig comprising primary joints and helper joints. The animation rig may be in a pose defined by spatial relationships between the primary joints and helper joints. The virtual character may be moving in the mixed reality environment. In some instances, the virtual character may be moving based on a comparison of interestingness values associated with elements in the mixed reality environment. The spatial relationship transformation associated with the movement may be indicated by movement information. In some embodiments, the movement information is received from a neural network.

Abstract: Systems and methods for rendering a video effect to a display are described. More specifically, video data and audio data are obtained. The video data is analyzed to determine one or more attachment points of a target object that appears in the video data. The audio data is analyzed to determine audio characteristics. A video effect associated with an animation to be added to the one or more attachment points is determined based on the audio characteristics. A rendered video is generated by applying the video effect to the video data.

Abstract: Implementations of the subject technology provide systems and methods for recording an extended reality experience in a way that allows the experience to be played back at a later time from a different viewpoint or perspective. This allows computer-generated content that was rendered for display to a user during the recording, to be re-rendered during playback at the correct time and location in the recording, but from a different perspective. In order to facilitate this type of viewer-centric playback, the recording includes a computer-generated content track that references resources for re-rendering the computer-generated content at each point in time in the recording.

Abstract: Systems, methods, and non-transitory computer readable media configured for managing content placement in extended reality environments are provided. In one implementation, at least one processor associated with the non-transitory computer readable medium may receive a request from an entity to place virtual content at a specific geographic location in at least one shared extended reality environment that includes a plurality of virtual objects, wherein the virtual objects are viewable by a plurality of users. Different content placement rules may define where content may be placed. Based on the obtained information, the processor may determine that the request meets or does not meet the specific content placement rule. The at least one processor may implement the specific content placement rule to prevent content from being displayed when the specific content placement rule is not met and may enable content to be displayed when the specific content placement rule is met.

Abstract: A method of evaluating a joint includes obtaining test data indicative of movement of the joint during a test of the joint, generating visualization data for a three-dimensional representation of the joint to be rendered via a display, generating plane data for a representation of a plane to be rendered via the display with the three-dimensional representation of the joint, the plane having a position and an orientation fixed relative to a bone of the joint, adjusting the visualization data to animate the three-dimensional representation to depict, via the display, the movement of the joint during the test, and adjusting the plane data to update the position and the orientation of the plane in accordance with the movement of the joint.

Abstract: Empowered by augmented reality (AR) technologies, the present disclosure allows a user to display virtual content in a physical reality and turn an AR-ready handheld mobile device into a dimension measuring tool. The present disclosure allows the user to first display a virtual container asset, with its actual size in physical reality, in any given configuration, and then create a virtual dimensional equivalent of an item-to-be-fit based on dimensional data captured by a 6-degree-of-freedom (6DoF) device or the like. Finally, the user can place the virtual item into the virtual container to evaluate the capacity and fit in the given configuration.

Abstract: A method may include receiving, via a processor, image data associated with a user's surrounding and generating, via the processor, a visualization that may include a virtual industrial automation device. The virtual industrial automation device may depict a virtual object within image data, and the virtual object may correspond to a physical industrial automation device. The method may include displaying, via the processor, the visualization via an electronic display and detecting, via the processor, a gesture in image data that may include the user's surrounding and the visualization. The gesture may be indicative of a request to move the virtual industrial automation device. The method may include tracking, via the processor, a user's movement, generating, via the processor, a visualization that may include an animation of the virtual industrial automation device moving based on the user's movement, and displaying, via the processor, the visualization via the electronic display.

Abstract: Disclosed are methods, systems, and computer program products for mixed-reality systems. These methods or systems determine a three-dimensional model for at least a portion of a physical environment in which a user is located; and present, by a spatial computing system, a mixed-reality representation to the user. In addition, these methods or systems determine a first object model for a first object in the mixed-reality representation and update, by the spatial computing system, the mixed-reality representation into an updated mixed-reality representation that reflects an interaction pertaining to the first object.

Abstract: Disclosed herein are systems and methods for sharing and synchronizing virtual content. A method may include receiving, from a host application via a wearable device comprising a transmissive display, a first data package comprising first data; identifying virtual content based on the first data; presenting a view of the virtual content via the transmissive display; receiving, via the wearable device, first user input directed at the virtual content; generating second data based on the first data and the first user input; sending, to the host application via the wearable device, a second data package comprising the second data, wherein the host application is configured to execute via one or more processors of a computer system remote to the wearable device and in communication with the wearable device.

Abstract: The present invention provides a method for automating the trimming of the boundary region of a three dimensional (3D) open surface mesh model comprising an inner mesh region delineated by boundary vertices and edges. In a particular embodiment, the present invention relates to a method for automating the trimming of an open surface mesh representing an intraoral region.

Abstract: Techniques image suitability determination to generate augmented reality/virtual reality (AR/VR) digital content are described. A two-dimensional digital image is received. Using machine-learning, a determination as to whether an object captured by the two-dimensional digital image is suitable for generating AR/VR digital content for display in an AR/VR environment is made. If the object is suitable, an indication is provided and an option to view the object in an AR/VR environment is provided. If the object is not suitable, a suggestion indicating why the object as captured is not suitable and/or how to correct the capture of the object in a subsequent digital image such that it is suitable for generating AR/VR digital content.

Abstract: Systems and methods for rendering a video effect to a display are described. More specifically, video data and audio data are obtained. The video data is analyzed to determine one or more attachment points of a target object that appears in the video data. The audio data is analyzed to determine audio characteristics. A video effect associated with an animation to be added to the one or more attachment points is determined based on the audio characteristics. A rendered video is generated by applying the video effect to the video data.

Abstract: Methods and systems for selecting an object or location in an extended reality (XR) environment or physical environment are described. A first origin, including a first position and a first direction, and a second origin, including a second position and a second direction, are obtained by at least one sensor. An intersection of a first ray, casted from the first origin, and a second ray, casted from the second origin, is determined. A selected object or selected location is identified, based on the determined intersection. An identification of the selected object or the selected location is outputted.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for receiving a monocular image that includes a depiction of a hand and extracting features of the monocular image using a plurality of machine learning techniques. The program and method further include modeling, based on the extracted features, a pose of the hand depicted in the monocular image by adjusting skeletal joint positions of a three-dimensional (3D) hand mesh using a trained graph convolutional neural network (CNN); modeling, based on the extracted features, a shape of the hand in the monocular image by adjusting blend shape values of the 3D hand mesh representing surface features of the hand depicted in the monocular image using the trained graph CNN; and generating, for display, the 3D hand mesh adjusted to model the pose and shape of the hand depicted in the monocular image.

Abstract: In some embodiments, casted content may be augmented with augmented reality content. In some embodiments, a dynamic value of moment related to a content portion of casted content may be obtained. The dynamic value of moment may be based on characteristic information related to the content portion of the casted content. An augmentation package may be generated, based on the dynamic value of moment, for the content portion. Based on a first entity being associated with the augmentation package, augmented reality content associated with the first entity may be selected to be presented with the content portion. In some embodiments, a modified content portion may be caused to be presented such that the augmented reality content associated with the first entity is presented with the content portion.