Abstract: A system and method for immersive telecommunications communication by tracking the movement of objects and/or persons with sensors. Movement tracking is then used to animate an avatar that represents the person or object. Movement may be tracked in real time, which at least reduces latency of communication. The sensors comprise any type of movement sensor which may be attached to a person and/or object for tracking motion, including but not limited to, an IMU (Inertial Measurement Unit), an accelerometer, a gyroscope or other such sensors.

Abstract: An apparatus configured to be worn on a head of a user, includes: a screen configured to present graphics to the user; a camera system configured to view an environment in which the user is located; and a processing unit configured to determine a map based at least in part on output(s) from the camera system, wherein the map is configured for use by the processing unit to localize the user with respect to the environment; wherein the processing unit of the apparatus is also configured to obtain a metric indicating a likelihood of success to localize the user using the map, and wherein the processing unit is configured to obtain the metric by computing the metric or by receiving the metric.

Abstract: An ortho-image creation method includes: first photographing of photographing a road; second photographing of photographing an area covered by the obstacle from an altitude lower than the obstacle by a second photographing apparatus, and obtaining a plurality of second photographed images; first coordinate acquisition of acquiring three-dimensional coordinates; second coordinate acquisition of acquiring three-dimensional coordinates of a second feature point located in the area covered by the obstacle and included in at least two of the plurality of second photographed images; and ortho-image creation of creating a corrected ortho-image obtained by correcting at least a part of the area covered by the obstacle in the road surface to an area not covered by the obstacle, on the basis of the plurality of first photographed images, the plurality of second photographed images, the three-dimensional coordinates of the first feature point, and the three-dimensional coordinates of the second feature point.

Abstract: A 3D objects training dataset is augmented by accessing a 3D point cloud representation of an object and by applying an augmentation routine on the point cloud to generate an augmented point cloud. The augmentation routine comprises randomly selecting an execution order of at least one of (i) adding a noise to the point cloud, (ii) applying a geometric transformation on the point cloud and (iii) applying a degradation on the point cloud. The randomly selected execution order of these operations on the point cloud is applied, and the augmented point cloud is added to the objects training dataset. A machine learning algorithm (MLA) is trained by inputting the 3D point cloud representation to generate an output, comparing the output with an expected label associated with the point cloud representation to determine a measure of error on the output, and iteratively adjusting various weights associated with nodes of the MLA.

Abstract: Systems are configured for performing GPS-based and sensor-based relocalization. During the relocalization, the systems are configured to obtain radio-based positioning data indicating an estimated position of the system within a mapped environment. The systems are also configured to identify, based on the estimated position, a subset of keyframes of a map of the mapped environment, wherein the map of the mapped environment includes a plurality of keyframes captured from a plurality of locations within the mapped environment, and the plurality of keyframes are associated with anchor points identified within the mapped environment. The systems are further configured to perform relocalization within the mapped environment based on the subset of keyframes.

Abstract: Techniques are described for using computing devices to perform automated operations related to providing visual information of multiple types in an integrated manner about a building or other defined area. The techniques may include generating and presenting a GUI (graphical user interface) on a client device that includes a computer model of the building's interior with one or more first types of information (e.g., in a first pane of the GUI), and simultaneously presenting other types of related information about the building interior (e.g., in additional separate GUI pane(s)) that is coordinated with the first type(s) of information being currently displayed. The computer model may be a 3D (three-dimensional) or 2.5D representation generated after the house is built and showing the actual house's interior (e.g., walls, furniture, etc.), and may be displayed to a user of a client computing device in a displayed GUI with various user-selectable controls.

Abstract: Systems and methods are disclosed for determining position and pose of as well as tracking an object in a physical environment based on the emission and sensing of light signals. The derived position, pose and tracking information may be used in a VR/AR environment. The disclosed systems and methods allow for the improved tracking of both active and passive devices. In addition, the disclosed systems and methods enable an arbitrary number of light sensors to be disposed on an object, thereby increasing accuracy and mitigating the effects of occlusion of certain light sensors. Position and pose estimates may be refined and tracked using a filter lattice responsive to changes in observed system states and/or settings. Further, data received from an inertial measurement unit may be used to increase tracking accuracy as well as position and pose determination itself.

Abstract: An image of a document is received from an image capture device, the image being in a format of an image file. At least one location of a user input field is automatically detected within the image based on patterns previously detected in a set of other images that were annotated to identify locations of user input fields within the individual images of the set. Coordinates are determined for the at least one location, and an electronic document is generated based on the received image. Generation of the electronic document includes addition of a software user input component at the location within the image with use of the coordinates, the software user input component configured to receive input from a user in electronic form.

Abstract: Generating data to provide an animated visual representation is disclosed. A method comprises receiving input data obtained by a first sensor system measuring information about at least one target person. One data unit is selected from a database comprising a plurality of the data units. Each data unit comprises information about a reference person in a reference state measured at a previous time by the first sensor system or by a second sensor system. The information in each data unit allows generation of an animated visual representation of the reference person in the reference state. The reference state is different for each of the data units. The selected data unit and the input data are used to generate output data usable to provide an animated visual representation corresponding to the target person and synchronized with activity of the target person measured by the first sensor system.

Abstract: A computing device spatially reconstructs a virtual feature surface in a mixed reality environment. The computing device detects addition of a raycast element to a virtual user space, maps multiple feature points detected from multiple video frames of a physical user space into a virtual user space, selecting at least three feature points from the multiple feature points that satisfy selection criteria applied in the virtual user space along a raycast axis of the raycast element in the virtual user space, and defines the virtual feature surface in the virtual user space using the at least three selected feature points. At least two of the at least three feature points are detected in different video frames.

Abstract: The disclosure provides computer systems for processing paths and a renderer that generates a stroked tessellation of a path. A data structure for processing the paths can be used, wherein the data structure is an array of indexed links that compactly encode a path. The position of one or more index values, such as a null index value, within an indexed link can encode a link's type. In one example, the computer system for processing links of a path includes one or more processing units to perform one or more operations including: (1) analyzing a data structure that encodes a link of a path, the data structure having multiple indices that refer to a control point coordinate array corresponding to the link, and (2) determining a type of the link based on a presence of at least one index null value for at least one of the indices.

Abstract: A method, computer-readable storage medium, and device for generating a character model. The method comprises: receiving an input image of a reference subject; processing the input image to generate a normalized image; identifying a set of features present in the normalized image, wherein each feature in the set of features corresponds to a portion of a head or body of the reference subject; for each feature in the set of features, processing at least a portion of the normalized image including the feature by a neural network model corresponding to the feature to generate a parameter vector corresponding to the feature; and combining the parameter vectors output by respective neural network models corresponding to respective features in the set of features to generate a parameterized character model corresponding to reference subject in the input image.

Abstract: Graphical style modification may be implemented using machine learning. A color accommodation module receives an image frame from a host system and generates a color-adapted version of the image frame. A Graphical Style Modification module generates a style adapted video stream by applying a style adapted from a target image frame to each image frame in a buffered video stream.

Abstract: Apparatus and methods for a smart glasses device are provided. The smart glasses device may execute a prediction model on video data captured by the smart glasses device to retrain the prediction model. The prediction model may be retrained in response to a data synchronization between an output of the prediction model and a gesture captured by the smart glasses device.

Abstract: An augmented reality/mixed reality system that provides an immersive user experience. That experience is provided with increased speed of update with low computational resources for occlusion data representing a user's hand that can be used to occlude virtual objects. The hand occlusion data are generated by computing a hand mesh from live depth sensor data. A small amount of data is processed by masking live depth data with a model in which a hand is simply represented by multiple segments identified from key-points. The hand occlusion data may be generated by predicting changes in a hand pose between a time of capture of the depth data and the time when the hand mesh will be used for occlusion processing. The hand mesh may be distorted to represent the hand in the predicted pose.

Abstract: The present technique relates to an information processing apparatus, and an information processing method in which data of content is acquired, and a first visual field image corresponding to a visual field of a first user is cut out from a content image based on the data of the content. In addition, visual field information representing a visual field of a second user viewing the content image is acquired. Furthermore, in a display apparatus, the first visual field image is displayed, and the visual field of the second user is displayed based on the visual field information of the second user. The present technique can be applied to, for example, a case where a plurality users views content, such as a spherical image and a free viewpoint image.

Abstract: Disclosed herein includes a system, a method, and a device for performing a convolution on data of a current layer of a neural network, including a plurality of channels arranged in a first order and partitioned into a plurality of first partitions according to the first order. Each first partition includes a result of a convolution on a corresponding partition of channels in data of a previous layer of the neural network. The device shifts the plurality of channels arranged in the first order to a second order, partition the shifted plurality of channels into a plurality of second partitions, according to the second order. For each of the plurality of second partitions, the device performs a convolution on channels of the shifted plurality of channels that are in the corresponding second partition.

Abstract: Systems and methods are provided for performing operations including: receiving, by a messaging application, an image captured by a front-facing camera of a computing device of a first user; displaying, by the messaging application, the image on a display screen of the computing device; receiving a request to send the image to a computing device of a second user; in response to receiving the request, generating a mirrored version of the image that was captured by the front-facing camera; and sending the mirrored version of the image to the second user.

Abstract: A computer-implemented method prevents physical interference between co-located users in virtual environments. The method includes identifying a first user associated with a first virtual reality (VR) system and a second user associated with a second VR system. The method also includes predicting a future movement of the first user to perform a first task in a primary virtual environment. The method further includes identifying, in response to the predicting the future movement of the first user, a first physical space needed for the first user. The method includes determining, in response to identifying the first physical space, a likely interference in a first physical location of the first user. The method also includes preventing the likely interference.

Abstract: The present disclosure advantageously provides a hardware accelerator for an artificial neural network (ANN), including a communication bus interface, a memory, a controller, and at least one processing engine (PE). The communication bus interface is configured to receive a plurality of finetuned weights associated with the ANN, receive input data, and transmit output data. The memory is configured to store the plurality of finetuned weights, the input data and the output data. The PE is configured to receive the input data, execute an ANN model using a plurality of fixed weights associated with the ANN and the plurality of finetuned weights, and generate the output data. Each finetuned weight corresponds to a fixed weight.