Abstract: A system for hand tracking may comprise a head mounted display wearable by a user and a hand tracking camera module attached to the head mounted display and comprising at least one of a pair of stereo cameras or a depth camera. The hand tracking camera module may be configured to capture images of at least one physical hand of the user. The head mounted display may be configured to render a virtual hand resembling the physical hand in a virtual environment for viewing by the user based at least on the images.

Abstract: A method for human-machine interaction includes monitoring a movement of an object by a sensor that detects positions of the object over time, generating a time-dependent velocity of the object based on the movement of the object, detecting a tapping event of the object tapping on a surface by detecting a sudden change of the time-dependent velocity, and determining a position of the object at a time when the tapping event occurs as a tapping position of the object.

Abstract: A system for computer vision is disclosed. The system may comprise a processor and a non-transitory computer-readable storage medium coupled to the processor. The non-transitory computer-readable storage medium may store instructions that, when executed by the processor, cause the system to perform a method. The method may comprise obtaining a first and a second images of at least a portion of an object, extracting a first and a second 2D contours of the portion of the object respectively from the first and second images, matching one or more first points on the first 2D contour with one or more second points on the second 2D contour to obtain a plurality of matched contour points and a plurality of mismatched contour points, and reconstructing a shape of the portion of the object based at least in part on at least a portion of the matched points and at least a portion of the mismatched contour points.

Abstract: A tracking system is disclosed. The system may comprise a processor and a non-transitory computer-readable storage medium coupled to the processor and storing instructions that, when executed by the processor, cause the system to perform a method. The method may comprise training a detection model and an extraction model, capturing one or more images of at least a portion of an object, detecting the portion of the object in each of the one or more images through the trained detection model, tracking the detected portion of the object in real-time, obtaining 2D positions of one or more locations on the tracked portion of the object through the trained extraction model, and obtaining 3D positions of the one or more locations on the tracked portion of the object based at least in part on the obtained 2D positions.

Abstract: A foldable apparatus is disclosed. The apparatus may comprise at least one camera configured to acquire an image of a physical environment, an orientation and position determination module configured to determine a change in orientation and/or position of the apparatus with respect to the physical environment based on the acquired image, a housing configured to hold the at least one camera and the orientation and position determination module, and a first strap attached to the housing and configured to attach the housing to a head of a user of the apparatus.

Abstract: A method is disclosed. The method may be implemented by an immersive and interactive multimedia generation system. The method may comprise projecting, by a first component of the system, a plurality of patterns to a physical environment in which a second component of the system is located, acquiring, by the second component of the system, a first image and a second image of at least a part of the physical environment, the first and second images including, respectively, first and second pixel data corresponding to at least some of the plurality of projected patterns, and determining, by the second component of the system, a change in at least one of an orientation or a position of the second component within the physical environment based on a relationship between the first and the second pixel data.

Abstract: An apparatus comprising a processor and a non-transitory computer-readable storage medium storing instructions, when executed by the processor, cause the apparatus to perform a method comprising determining a first and second set of coordinates, associated with a first and second feature, respectively, in one or more first and second images, respectively, the first and second images are captured from a first and second position and/or orientation relative to the physical environment, respectively, re-projecting the first set of coordinates to one or more 2D spaces associated with the second images, comparing the re-projected first set of coordinates with the second set of coordinates in at least one of position closeness, feature closeness, or stereo constraints to determine a correspondence between the first and second features and determining a change between the first and second orientations and/or positions with respect to the physical environment based on the determined correspondence.

Abstract: A method for human-machine interaction includes monitoring a movement of an object by a sensor that detects positions of the object over time, generating a time-dependent velocity of the object based on the movement of the object, detecting a tapping event of the object tapping on a surface by detecting a sudden change of the time-dependent velocity, and determining a position of the object at a time when the tapping event occurs as a tapping position of the object.

Abstract: An apparatus is disclosed. The apparatus comprises an optical sensing system that comprises at least one camera, the at least one camera being configured to acquire an image of a physical environment. The apparatus further comprises a processing system. The processing system comprises an orientation and position determination module configured to detect salient features from the image, and determine a change in orientation and/or position of the apparatus with respect to the physical environment based on the detected salient features. The processing system also comprises a rendering module configured to determine a rendering of the physical environment based on the image and on the determined change in orientation and/or position of the apparatus, and provide data related to the rendering of the physical environment to a display system.

Abstract: A method for three-dimensional (3D) sensing. The method includes obtaining a first two-dimensional (2D) skeleton of an object, obtaining a second 2D skeleton of the object different from the first 2D skeleton, and calculating a 3D skeleton of the object based on the first and second 2D skeletons.