Abstract: A system and method for generating input data from a pose estimate for a pose (position and orientation) of a manipulated object operated in a three-dimensional environment that offers optical features. The manipulated object has an on-board photodetector for providing light data and an on-board auxiliary motion detection component for providing relative motion data indicative of a change in an orientation, a change in position or both (relative change in pose). A processor in communication with the on-board photodetector and auxiliary motion detection component uses light data to determine an absolute pose estimate at times ti and relative motion data to determine a relative pose change. The processor deploys a technique that combines the absolute pose estimate from the light data and the relative pose change from the relative motion data to provide the pose estimate at an application request time tr.

Abstract: The present invention discloses systems and methods for both viewing and interacting with a virtual reality (VR), an augmented reality (AR) or a mixed reality (MR). More specifically, the systems and methods allow the user to interact with aspects of such realities including virtual items presented in such realities or within such environments by manipulating a control device that has an inside-out camera mounted on-board. The apparatus or system uses two distinct representations including a reduced representation in determining the pose of the control device and uses these representations to compute an interactive pose portion of the control device to be used for interacting with the virtual item. The reduced representation is consonant with a constrained motion of the control device.

Abstract: The present invention discloses systems and methods for both viewing and interacting with a virtual reality (VR), an augmented reality (AR) or a mixed reality (MR). More specifically, the systems and methods allow the user to interact with aspects of such realities including virtual items presented in such realities or within such environments by manipulating a control device that has an inside-out camera mounted on-board. The apparatus or system uses two distinct representations including a reduced representation in determining the pose of the control device and uses these representations to compute an interactive pose portion of the control device to be used for interacting with the virtual item. The reduced representation is consonant with a constrained motion of the control device.

Abstract: A manipulated object using high optical contrast features and an on-board photodetector that detects light produced, reflected or scattered by the features and outputs data indicative thereof. The manipulated object uses one or more controllers to determine its position and/or orientation, including full pose, in the real three-dimensional environment based on data from the photodetector. Data from one or more auxiliary motion sensing devices, e.g., a relative motion sensor such as an inertial device or other auxiliary motion device relying on acoustics, optics or electromagnetic waves within or outside the visible spectrum, can be used to supplement the position and/or orientation data from the photodetector. The manipulated object can be embodied by any suitable device manipulated by a user, including a tablet computer or a phone.

Abstract: A remotely controlled object (e.g., an aircraft) or a wearable article (e.g., virtual or augmented reality glasses) using high optical contrast features and a photodetector that detects light produced, reflected or scattered by the features and outputs data indicative thereof. The remotely controlled object or wearable article uses one or more controllers to determine its position and/or orientation in the real three-dimensional environment based on data from the photodetector. Data from one or more auxiliary motion sensing devices, e.g., a relative motion sensor such as an inertial device or other auxiliary motion device relying on acoustics, optics or electromagnetic waves within or outside the visible spectrum, can be used to supplement the position and/or orientation data from the photodetector.

Abstract: Efficient techniques of recovering the pose of an optical apparatus exploiting structural redundancies due the conditioned motion of an apparatus are disclosed. The techniques are based on determining a reduced homography consonant to the conditioned motion of the optical apparatus. The optical apparatus comprises an optical sensor on which space points are imaged as measured image points. The reduced homography is based on a reduced representation of the space points, obtained by exploiting the structural redundancy in the measured image points due to the conditioned motion. The reduced representation consonant with the conditioned motion is defined by rays in homogeneous coordinates and contained in a projective plane of the optical sensor.

Abstract: A wearable article, such as glasses for a virtual reality program or glasses for an augmented reality application, using light sources and a photodetector that detects their light and outputs data indicative of the detected light. The wearable article uses one or more controllers to determine its position and/or orientation in the environment based on the data output by the photodetector. Data from one or more auxiliary motion sensing devices, e.g., a relative motion sensor such as an inertial device or other auxiliary motion device relying on acoustics, optics or electromagnetic waves within or outside the visible spectrum, can be used to supplement the position and/or orientation data from the photodetector.

Abstract: The present invention relates to interfaces and methods for producing input for software applications based on the absolute pose of an item manipulated or worn by a user in a three-dimensional environment. Absolute pose in the sense of the present invention means both the position and the orientation of the item as described in a stable frame defined in that three-dimensional environment. The invention describes how to recover the absolute pose with optical hardware and methods, and how to map at least one of the recovered absolute pose parameters to the three translational and three rotational degrees of freedom available to the item to generate useful input. The applications that can most benefit from the interfaces and methods of the invention involve 3D virtual spaces including augmented reality and mixed reality environments.

Abstract: A reduced homography H for an optical apparatus to recover pose parameters from imaged space points Pi using an optical sensor. The electromagnetic radiation from the space points Pi is recorded on the optical sensor at measured image coordinates. A structural uncertainty introduced in the measured image points is determined and a reduced representation of the measured image points is selected based on the type of structural uncertainty. The reduced representation includes rays {circumflex over (r)}i defined in homogeneous coordinates and contained in a projective plane of the optical apparatus. At least one pose parameter of the optical apparatus is then estimated by applying the reduced homography H and by applying a condition on the motion of the optical apparatus, the condition being consonant with the reduced representation employed in the reduced homography H.

Abstract: A reduced homography H for an optical apparatus to recover pose parameters from imaged space points Pi using an optical sensor. The electromagnetic radiation from the space points Pi is recorded on the optical sensor at measured image coordinates. A structural uncertainty introduced in the measured image points is determined and a reduced representation of the measured image points is selected based on the type of structural uncertainty. The reduced representation includes rays {circumflex over (r)}i defined in homogeneous coordinates and contained in a projective plane of the optical apparatus. At least one pose parameter of the optical apparatus is then estimated by applying the reduced homography H and by applying a condition on the motion of the optical apparatus, the condition being consonant with the reduced representation employed in the reduced homography H.

Abstract: A system with a remote control or wand equipped with a relative motion sensor that outputs data indicative of a change in position. The system has one or more light sources and a photodetector that detects their light and outputs data indicative of the detected light. One or more controllers are used to determine the absolute position of the wand based on the data output by the relative motion sensor and by the photodetector. The wand's absolute pose is determined from the data and includes the absolute position of a reference point on the wand and the wand's absolute orientation. A display is used to show an image defined by two orthogonal axes, e.g., those of world coordinates (Xo,Yo,Zo). The display is rendered as a function of absolute position of the wand in or along a third orthogonal axis.

Abstract: A system with a remote control or wand equipped with a relative motion sensor that outputs data indicative of a change in position. The system has one or more light sources and a photodetector that detects their light and outputs data indicative of the detected light. One or more controllers are used to determine the absolute position of the wand based on the data output by the relative motion sensor and by the photodetector. The wand's absolute pose is determined from the data and includes the absolute position of a reference point on the wand and the wand's absolute orientation. A display is used to show an image defined by two orthogonal axes, e.g., those of world coordinates (Xo,Yo,Zo). The display is rendered as a function of absolute position of the wand in or along a third orthogonal axis.

Abstract: A system that has a remote control, e.g., a wand, equipped with a relative motion sensor that outputs data indicative of a change in position of the wand. The system also has one or more light sources and a photodetector that detects their light and outputs data indicative of the detected light. The system uses one or more controllers to determine the absolute position of the wand based on the data output by the relative motion sensor and by the photodetector. The data enables determination of the absolute pose of the wand, which includes the absolute position of a reference point chosen on the wand and the absolute orientation of the wand. To properly express the absolute parameters of position and/or orientation of the wand a reference location is chosen with respect to which the calculations are performed. The system is coupled to a display that shows an image defined by a first and second orthogonal axes such as two axes belonging to world coordinates (Xo,Yo,Zo).

Abstract: An apparatus and method for processing pose data derived from a pose of an elongate object such as, for example, a jotting implement, a pointer, a robotic arm or a cane. The elongate object has a tip contacting a plane surface with one or more invariant features. The pose of the elongate object is measured optically from on-board by an optical measuring system with the aid of the invariant feature. The pose is used for preparing a corresponding pose data and a subset of the pose data is identified and transmitted to an application such as a user application, where the subset can serve as command data or input data. Since the elongate object moves while its tip is contacting the surface the pose is measured periodically at sufficiently frequent measurement times ti to describe the motion at a desired temporal resolution. The subset can include all or a portion of the orientation data that describe the orientation of the elongate object in space and/or position data of the tip on the surface.

Abstract: A reduced homography H for an optical apparatus to recover pose parameters from imaged space points Pi using an optical sensor. The electromagnetic radiation from the space points Pi is recorded on the optical sensor at measured image coordinates. A structural uncertainty introduced in the measured image points is determined and a reduced representation of the measured image points is selected based on the type of structural uncertainty. The reduced representation includes rays {circumflex over (r)}i defined in homogeneous coordinates and contained in a projective plane of the optical apparatus. At least one pose parameter of the optical apparatus is then estimated by applying the reduced homography H and by applying a condition on the motion of the optical apparatus, the condition being consonant with the reduced representation employed in the reduced homography H.

Abstract: A system that has a remote control, e.g., a wand, equipped with a relative motion sensor that outputs data indicative of a change in position of the wand. The system also has one or more light sources and a photodetector that detects their light and outputs data indicative of the detected light. The system uses one or more controllers to determine the absolute position of the wand based on the data output by the relative motion sensor and by the photodetector. The data enables determination of the absolute pose of the wand, which includes the absolute position of a reference point chosen on the wand and the absolute orientation of the wand. To properly express the absolute parameters of position and/or orientation of the wand a reference location is chosen with respect to which the calculations are performed. The system is coupled to a display that shows an image defined by a first and second orthogonal axes such as two axes belonging to world coordinates (Xo,Yo,Zo).

Abstract: An apparatus and method for optically inferring an absolute pose of a manipulated object in a real three-dimensional environment from on-board the object with the aid of an on-board optical measuring arrangement. At least one invariant feature located in the environment is used by the arrangement for inferring the absolute pose. The inferred absolute pose is expressed with absolute pose data (?, ?, ?, x, y, z) that represents Euler rotated object coordinates expressed in world coordinates (Xo, Yo, Zo) with respect to a reference location, such as, for example, the world origin. Other conventions for expressing absolute pose data in three-dimensional space and representing all six degrees of freedom (three translational degrees of freedom and three rotational degrees of freedom) are also supported. Irrespective of format, a processor prepares the absolute pose data and identifies a subset that may contain all or fewer than all absolute pose parameters.

Abstract: The present invention relates to an optical navigation system for determining a pose, which includes the position and orientation of an object in an environment. The optical navigation system uses a number of beacons affixed at known locations in the environment to provide electromagnetic radiation in a sequenced pattern. An on-board optic images the radiation from the beacons onto an on-board centroid sensing device to obtain an imaged distribution of the radiation on the on-board centroid sensing device. The centroid sensing device determines the centroid of the imaged distribution and provides centroid information to a navigation unit for determining the pose of the object from the centroid. The navigation system is particularly well-suited for navigating hand-held objects.

Abstract: The present invention relates to a jotting implement that infers hand-jotted information from viewing a jotting surface and environmental landmarks. The hand-jotted information is any information marked on the jotting surface as a result of writing, jotting, drawing, sketching or otherwise operating on the jotting surface as well as hand-generated information produced by moving the implement in the air with respect to the jotting surface. The jotting implement has a nib for jotting and an arrangement for determining when the nib is jotting on the jotting surface. Further, the implement has an optical unit for viewing an environmental landmark and the jotting surface. The implement has a processing unit for receiving optical data of the environmental landmark and the jotting surface from the optical unit and determining from the optical data the physical coordinates of the nib with respect to the jotting surface.

Abstract: A dual viewpoint solid catadioptric lens has a first spherical refractive surface S1 having a center C1 located on an optical axis of the lens and having a radius r1, and a second spherical refractive surface S2 having a center C2 located on the optical axis of the lens and having a radius r2<r1. The lens also has a first ellipsoidal reflective surface E1 with foci F1 and F1? on the optical axis of the lens, and a second ellipsoidal partially reflective surface E2 having foci F2 and F2? on the optical axis of the lens. Focus F1 coincides with C1, focus F2 conincides with C2, and focus F1? coincides with F2?. The points C1 and C2 provide dual viewpoints for the lens, which may be used in a variety of imaging applications.