Abstract: A method for representing virtual information in a view of a real environment is provided that includes: providing a system setup including at least one display device, wherein the system setup is adapted for blending in virtual information on the display device in at least part of the view, determining a position and orientation of a viewing point relative to at least one component of the real environment, providing a geometry model of the real environment, providing at least one item of virtual information and a position of the at least one item of virtual information, determining whether the position of the item of virtual information is inside a 2D or 3D geometrical shape, determining a criterion which is indicative of whether the built-in real object is at least partially visible or non-visible in the view of the real environment, and blending in the at least one item of virtual information on the display device in at least part of the view of the real environment.

Abstract: Generating a 3D model may include determining, based on sensor data from a touch sensor on a first device, a touch event, wherein the touch event comprises a touch on the first device by a touching object. Generating a 3D model may also include, in response to a touch event, obtaining a first image of the touching object by a first camera of the first device, and obtaining, from a second device, a second image of the touching object, wherein the first image of the touching object captures a first view of the touching object, and wherein the second image of the touching object captures a second view of the touching object. A model of the touching object is generated based on the first image and the second image.

Abstract: Touch detection includes obtaining image data for a finger and a target surface, and determining a touch region in the image data on which the fingertip makes contact with the target surface. A pointing direction of the finger in the image data is determined, and a target region is estimated on the target surface based on the touch region and the pointing direction, wherein the touch region comprises a portion of the target surface that extends from the touch region in a direction of the pointing direction.

Abstract: A method for providing position information for controlling at least one function of an environment provided with an environment coordinate system comprises the steps of receiving image information associated with at least one image of at least one part of the environment captured by a camera of a mobile device, the mobile device being separate from the environment, determining a position associated with the mobile device in the environment coordinate system according to the image information associated with the at least one image, and providing position information indicative of the position associated with the mobile device in the environment coordinate system to control at least one function of the environment according to the position. There is also provided a method and system which provides movement information indicative of a movement associated with the mobile device relative to the environment coordinate system to control at least one function of the environment according to the movement.

Abstract: The present disclosure relates generally to remote touch detection. In some examples, a first electronic device obtains first image data and second image data about an input, and performs an operation based on the input in accordance with a determination that a set of one or more criteria is met based on the first image data and the second image data. In some examples, a first electronic device causes emission of infrared light by an infrared source of a second electronic device, obtains image data about an input, and performs an operation based on the input in accordance with a determination that a set of one or more criteria is met based on the image data.

Abstract: A method for representing virtual information in a view of a real environment is provided that includes: providing a system setup including at least one display device, wherein the system setup is adapted for blending in virtual information on the display device in at least part of the view, determining a position and orientation of a viewing point relative to at least one component of the real environment, providing a geometry model of the real environment, providing at least one item of virtual information and a position of the at least one item of virtual information, determining whether the position of the item of virtual information is inside a 2D or 3D geometrical shape, determining a criterion which is indicative of whether the built-in real object is at least partially visible or non-visible in the view of the real environment, and blending in the at least one item of virtual information on the display device in at least part of the view of the real environment.

Abstract: In one implementation, a device has a processor, a projector, a first infrared (IR) sensor, a second IR sensor, and instructions stored on a computer-readable medium that are executed by the processor to estimate the sensor-to-sensor extrinsic parameters. The projector projects IR pattern elements onto an environment surface. The first sensor captures a first image including first IR pattern elements corresponding to the projected IR pattern elements and the device estimates 3D positions for first IR pattern elements. The second IR sensor captures a second image including second IR pattern elements corresponding to the projected IR pattern elements and the device matches the first IR pattern elements and the second IR pattern elements. Based on this matching, the device estimates a second extrinsic parameter corresponding to a spatial relationship between the first IR sensor and the second IR sensor.

Abstract: The present disclosure relates to a method of providing at least one image of at least one real object captured by at least one scene camera of a plurality of scene cameras mounted on a vehicle. The method includes: providing camera poses of respective scene cameras of the plurality of scene cameras relative to a reference coordinate system associated with the vehicle, providing user attention data related to a user captured by an information capturing device, providing at least one attention direction relative to the reference coordinate system from the user attention data, determining at least one of the scene cameras among the plurality of scene cameras according to the at least one attention direction and the respective camera pose of the at least one of the scene cameras, and providing at least one image of at least one real object captured by the at least one of the scene cameras.

Abstract: The present disclosure relates generally to remote touch detection. In some examples, a first electronic device obtains first image data and second image data about an input, and performs an operation based on the input in accordance with a determination that a set of one or more criteria is met based on the first image data and the second image data. In some examples, a first electronic device causes emission of infrared light by an infrared source of a second electronic device, obtains image data about an input, and performs an operation based on the input in accordance with a determination that a set of one or more criteria is met based on the image data.

Abstract: The present disclosure relates to a method of providing at least one image of at least one real object captured by at least one scene camera of a plurality of scene cameras mounted on a vehicle. The method includes: providing camera poses of respective scene cameras of the plurality of scene cameras relative to a reference coordinate system associated with the vehicle, providing user attention data related to a user captured by an information capturing device, providing at least one attention direction relative to the reference coordinate system from the user attention data, determining at least one of the scene cameras among the plurality of scene cameras according to the at least one attention direction and the respective camera pose of the at least one of the scene cameras, and providing at least one image of at least one real object captured by the at least one of the scene cameras.

Abstract: The disclosure relates to a method and system for presenting at least part of an image of a real object in a view of a real environment, comprising providing a first image of at least part of a real object captured by a first camera, determining at least part of the real object in the first image as an object image area, determining a first 3D plane relative to the first camera, the first camera being at a position where the first image is captured, providing at least one image feature related to the real object in the first image, providing at least one first ray passing an optical center of the first camera being at a position where the first image is captured and the at least one image feature, determining, according to a first plane normal direction of the first 3D plane, at least one first angle between the first 3D plane and the at least one first ray, providing a second image of a real environment captured by a second camera, determining a second 3D plane relative to the second camera, the second camer

Abstract: Touch detection may include determining, based on data from an IMU on a first device that monitors movement of a touching object, a touch event, wherein the touch event indicates contact between the touching object and a surface, obtaining a depth image captured by a second device, comprising a touch image, determining a touch point of the object based on the touch image, and providing a response based on the touch point of the object and the touched surface.

Abstract: Generating a 3D model may include determining, based on sensor data from a touch sensor on a first device, a touch event, wherein the touch event comprises a touch on the first device by a touching object. Generating a 3D model may also include, in response to a touch event, obtaining a first image of the touching object by a first camera of the first device, and obtaining, from a second device, a second image of the touching object, wherein the first image of the touching object captures a first view of the touching object, and wherein the second image of the touching object captures a second view of the touching object. A model of the touching object is generated based on the first image and the second image.

Abstract: The present disclosure relates generally to remote touch detection. In some examples, a first electronic device obtains first image data and second image data about an input, and performs an operation based on the input in accordance with a determination that a set of one or more criteria is met based on the first image data and the second image data. In some examples, a first electronic device causes emission of infrared light by an infrared source of a second electronic device, obtains image data about an input, and performs an operation based on the input in accordance with a determination that a set of one or more criteria is met based on the image data.

Abstract: Systems, methods, and computer readable media to improve the operation of detecting contact between a finger or other object and a surface are described. In general, techniques disclosed herein utilize a depth map to identify an object and a surface, and a classifier to determine when the object is touching the surface. Unlike the prior art, a measure of the object's “distance” is made relative to the surface and not the camera(s) thereby providing some measure of invariance with respect to camera pose. The object-surface distance measure can be used to construct an identifier or “feature vector” that, when applied to a classifier, generates an output indicative of whether the object is touching the surface. The classifier may be based on machine learning and can be trained off-line before run-time operations are commenced. In some embodiments, temporal filtering may be used to improve surface detection operations.

Abstract: The present disclosure relates to a method of providing at least one image of at least one real object captured by at least one scene camera of a plurality of scene cameras mounted on a vehicle. The method includes: providing camera poses of respective scene cameras of the plurality of scene cameras relative to a reference coordinate system associated with the vehicle, providing user attention data related to a user captured by an information capturing device, providing at least one attention direction relative to the reference coordinate system from the user attention data, determining at least one of the scene cameras among the plurality of scene cameras according to the at least one attention direction and the respective camera pose of the at least one of the scene cameras, and providing at least one image of at least one real object captured by the at least one of the scene cameras.

Abstract: The invention is related to a method and system for determining a pose of a first camera, comprising providing or receiving a spatial relationship (Rvc 1) between a visual content displayed on a display device and the first camera, receiving image information associated with an image (B1) of at least part of the displayed visual content captured by a second camera, and determining a pose of the first camera according to the image information associated with the image (B1) and the spatial relationship (Rvc1).

Abstract: The present disclosure relates to a method of providing at least one image of at least one real object captured by at least one scene camera of a plurality of scene cameras mounted on a vehicle. The method includes: providing camera poses of respective scene cameras of the plurality of scene cameras relative to a reference coordinate system associated with the vehicle, providing user attention data related to a user captured by an information capturing device, providing at least one attention direction relative to the reference coordinate system from the user attention data, determining at least one of the scene cameras among the plurality of scene cameras according to the at least one attention direction and the respective camera pose of the at least one of the scene cameras, and providing at least one image of at least one real object captured by the at least one of the scene cameras.

Abstract: A method for providing position information for controlling at least one function of an environment provided with an environment coordinate system comprises the steps of receiving image information associated with at least one image of at least one part of the environment captured by a camera of a mobile device, the mobile device being separate from the environment, determining a position associated with the mobile device in the environment coordinate system according to the image information associated with the at least one image, and providing position information indicative of the position associated with the mobile device in the environment coordinate system to control at least one function of the environment according to the position. There is also provided a method and system which provides movement information indicative of a movement associated with the mobile device relative to the environment coordinate system to control at least one function of the environment according to the movement.

Abstract: Three dimensional model construction includes obtaining a first image of an object captured by a first camera facing a first direction, obtaining a second image of the object captured by a second camera facing a second direction, wherein the second camera captures the second image of the object as a reflection in a mirror, determining a virtual baseline distance between the first camera and a device including the second camera based on a location of the first camera, a known spatial relationship between the first camera and the device, and a reflected distance of the second camera, and generating at least a partial 3D model of the first object using the first image, the second image, and the virtual baseline distance.