Abstract: Triggering a state change includes displaying a first version of a series of frames based on a first setup configuration, obtaining a second setup configuration for the series of frames, in response to obtaining a second setup configuration, monitoring for a change in an eye status, and in response to detecting a change in the eye status, displaying a second version of the series of frames based on the second setup configuration.

Abstract: A method of providing a descriptor for at least one feature of an image comprises the steps of providing an image captured by a capturing device and extracting at least one feature from the image, and assigning a descriptor to the at least one feature, the descriptor depending on at least one parameter which is indicative of an orientation, wherein the at least one parameter is determined from the orientation of the capturing device measured by a tracking system. The invention also relates to a method of matching features of two or more images.

Abstract: A VR system for vehicles that may implement methods that address problems with vehicles in motion that may result in motion sickness for passengers. The VR system may provide virtual views that match visual cues with the physical motions that a passenger experiences. The VR system may provide immersive VR experiences by replacing the view of the real world with virtual environments. Active vehicle systems and/or vehicle control systems may be integrated with the VR system to provide physical effects with the virtual experiences. The virtual environments may be altered to accommodate a passenger upon determining that the passenger is prone to or is exhibiting signs of motion sickness.

Abstract: The invention provides methods of detecting and describing features from an intensity image. In one of several aspects, the method comprises the steps of providing an intensity image captured by a capturing device, providing a method for determining a depth of at least one element in the intensity image, in a feature detection process detecting at least one feature in the intensity image, wherein the feature detection is performed by processing image intensity information of the intensity image at a scale which depends on the depth of at least one element in the intensity image, and providing a feature descriptor of the at least one detected feature. For example, the feature descriptor contains at least one first parameter based on information provided by the intensity image and at least one second parameter which is indicative of the scale.

Abstract: A VR system for vehicles that may implement methods that address problems with vehicles in motion that may result in motion sickness for passengers. The VR system may provide augmented or virtual views that match visual cues with the physical motions that a passenger experiences. The VR system may project virtual content so that the content appears as a distant object stabilized or fixed in the external environment. The VR system may aid in productivity, as passengers may perform work while riding in the vehicle without experiencing motion sickness. In addition, the VR system may provide enhanced virtual experiences to passengers in moving vehicles by matching accelerations and motions of the vehicle to accelerations and motions in the virtual experiences.

Abstract: In one implementation, a method is disclosed for providing visual coherency between virtual objects and a physical environment. The method includes obtaining, at an electronic device, first content depicting a physical surface in the physical environment using an image sensor of the electronic device. An extrinsic property exhibited by the physical surface is determined based on the first content using a visual coherency model. Second content representing a virtual object is generated based on the extrinsic property to present on a display.

Abstract: Systems, methods, and computer readable media to improve the operation of detecting contact between a finger or other object and a surface are described. In general, techniques disclosed herein utilize a depth map to identify an object and a surface, and a classifier to determine when the object is touching the surface. Unlike the prior art, a measure of the object's “distance” is made relative to the surface and not the camera(s) thereby providing some measure of invariance with respect to camera pose. The object-surface distance measure can be used to construct an identifier or “feature vector” that, when applied to a classifier, generates an output indicative of whether the object is touching the surface. The classifier may be based on machine learning and can be trained off-line before run-time operations are commenced. In some embodiments, temporal filtering may be used to improve surface detection operations.

Abstract: Determining spatial coordinates of a 3D reconstruction includes obtaining, from a first camera system, a first image comprising a first real object, obtaining, from a second camera system, a second image comprising a second real object associated with known geometric properties, wherein the first camera system and the second camera system have a known spatial relationship, and determining a scale of the face based on the second image and the known geometric properties of the at least part of the second real object. Determining the spatial coordinates of the 3D reconstruction also includes determining a pose of the first and second camera systems, and determining the spatial coordinates based on the pose of the first camera system and the scale of the at least part of the second real object.

Abstract: A method of providing a set of feature descriptors configured to be used in matching an object in an image of a camera is provided. The method includes: a) providing at least two images of a first object; b) extracting in at least two of the images at least one feature from the respective image, c) providing at least one descriptor for an extracted feature, and storing the descriptors; d) matching descriptors in the first set of descriptors; e) computing a score parameter based on the result of the matching process; f) selecting at least one descriptor based on its score parameter; g) adding the selected descriptor(s) to a second set of descriptors; and h) updating the score parameter of descriptors in the first set based on a selection process and to the result of the matching process.

Abstract: A method of providing a set of feature descriptors configured to be used in matching an object in an image of a camera is provided. The method includes: a) providing at least two images of a first object; b) extracting in at least two of the images at least one feature from the respective image, c) providing at least one descriptor for an extracted feature, and storing the descriptors; d) matching descriptors in the first set of descriptors; e) computing a score parameter based on the result of the matching process; f) selecting at least one descriptor based on its score parameter; g) adding the selected descriptor(s) to a second set of descriptors; and h) updating the score parameter of descriptors in the first set based on a selection process and to the result of the matching process.

Abstract: Various implementations disclosed herein include devices, systems, and methods that render a reflective surface of a computer-generated reality (“CGR”) object based on synthesis in a CGR environment. In order to render a reflective surface of the CGR object, one exemplary implementation involves synthesizing an environment map of a CGR environment representing a portion of a physical scene based on observed characteristics of the physical scene. In an implementation, generation of a complete environment map includes identifying pixels of the environment map with no corresponding texture and generating synthesized texture based on textural information associated with one or more camera images of the physical scene. In an implementation, a CGR object is rendered in the CGR environment, wherein an appearance of a reflective surface of the CGR object is determined based on the complete environment map of the CGR environment.

Abstract: A VR system for vehicles that may implement methods that address problems with vehicles in motion that may result in motion sickness for passengers. The VR system may provide augmented or virtual views that match visual cues with the physical motions that a passenger experiences. The VR system may project virtual content so that the content appears as a distant object stabilized or fixed in the external environment. The VR system may aid in productivity, as passengers may perform work while riding in the vehicle without experiencing motion sickness. In addition, the VR system may provide enhanced virtual experiences to passengers in moving vehicles by matching accelerations and motions of the vehicle to accelerations and motions in the virtual experiences.

Abstract: The invention, in one aspect, is related to a method of determining spatial coordinates of a 3D reconstruction of at least part of a first real object at absolute spatial scale comprising the steps of: receiving image information of a first image including at least part of the first real object captured with a first camera, of a third image including the at least part of the first real object captured with a third camera, of a second image including at least a first part of a second real object captured with a second camera, and of a fourth image including at least a second part of the second real object captured with a fourth camera, wherein the frustum of the first camera and the frustum of the second camera do not overlap and wherein the frustum of the third camera and the frustum of the fourth camera do not overlap, and providing a first spatial transformation between the first camera and the second camera and providing a second spatial transformation between the third camera and the fourth camera, provid

Abstract: A method of providing a set of feature descriptors configured to be used in matching an object in an image of a camera is provided. The method includes: a) providing at least two images of a first object; b) extracting in at least two of the images at least one feature from the respective image, c) providing at least one descriptor for an extracted feature, and storing the descriptors; d) matching descriptors in the first set of descriptors; e) computing a score parameter based on the result of the matching process; f) selecting at least one descriptor based on its score parameter; g) adding the selected descriptor(s) to a second set of descriptors; and h) updating the score parameter of descriptors in the first set based on a selection process and to the result of the matching process.

Abstract: A method of generating at least one image of a real environment comprises providing at least one environment property related to at least part of the real environment, providing at least one virtual object property related to a virtual object, determining at least one imaging parameter according to the at least one provided virtual object property and the at least one provided environment property, and generating at least one image of the real environment representing information about light leaving the real environment according to the determined at least one imaging parameter, wherein the light leaving the real environment is measured by at least one camera.

Abstract: A method of providing a set of feature descriptors configured to be used in matching an object in an image of a camera is provided. The method includes: a) providing at least two images of a first object; b) extracting in at least two of the images at least one feature from the respective image, c) providing at least one descriptor for an extracted feature, and storing the descriptors; d) matching descriptors in the first set of descriptors; e) computing a score parameter based on the result of the matching process; f) selecting at least one descriptor based on its score parameter; g) adding the selected descriptor(s) to a second set of descriptors; and h) updating the score parameter of descriptors in the first set based on a selection process and to the result of the matching process.

Abstract: The invention is related to a method of determining a model which describes at least part of a real object, which comprises the steps of: providing a first image of at least part of the real object to a first processing device, wherein the first image is captured by a camera, sending at least part of the first image to a second processing device which is physically separate from the first processing device, determining on the second processing device a registration between the first image and a first model provided on the second processing device according to the at least part of the first image received at the second processing device, determining geometric information for at least one pixel in the first image according to the registration, sending the geometric information from the second processing device to the first processing device, and using at least part of the geometric information received at the first processing device together with at least part of the first image to determine on the first proces

Abstract: Systems, methods, and computer readable media to improve the operation of detecting contact between a finger or other object and a surface are described. In general, techniques disclosed herein utilize a depth map to identify an object and a surface, and a classifier to determine when the object is touching the surface. Unlike the prior art, a measure of the object's “distance” is made relative to the surface and not the camera(s) thereby providing some measure of invariance with respect to camera pose. The object-surface distance measure can be used to construct an identifier or “feature vector” that, when applied to a classifier, generates an output indicative of whether the object is touching the surface. The classifier may be based on machine learning and can be trained off-line before run-time operations are commenced. In some embodiments, temporal filtering may be used to improve surface detection operations.

Abstract: An augmented safety restraint system includes a first restraint operable to secure a first passenger. The first restraint has an exposed surface facing away from a body of the first passenger. A gesture-sensing device is disposed on the exposed surface of the first restraint and is operable to receive an input from the first passenger.

Abstract: A method of providing a set of feature descriptors configured to be used in matching an object in an image of a camera is provided. The method includes: a) providing at least two images of a first object; b) extracting in at least two of the images at least one feature from the respective image, c) providing at least one descriptor for an extracted feature, and storing the descriptors; d) matching descriptors in the first set of descriptors; e) computing a score parameter based on the result of the matching process; f) selecting at least one descriptor based on its score parameter; g) adding the selected descriptor(s) to a second set of descriptors; and h) updating the score parameter of descriptors in the first set based on a selection process and to the result of the matching process.