Abstract: A system and method for high integrity head pose estimation receives a 2D image including medium-density fiducials and low-density fiducials. The system then detects the medium-density fiducials and compares them with a 3D constellation to form an initial head pose estimate. The initial head pose estimate is then checked for possible alternative correspondences. A selection from a set of candidate initial head pose estimates is made to determine. The selected head pose estimate then has search areas established for the medium-density fiducials and low-density fiducials. The system then determines whether or not the head pose is valid and accurate based on whether or not the fiducials appear in their respective search areas.

Abstract: A system and method for modifying symbology displayed on a head worn display (HWD) based on a quality-of-service value is disclosed. The system includes a head worn display configured to display symbology, a tracking camera, a controller, and a tracking processing unit configured to receive inputs form the tracking camera and transmit a signal quality value to the controller. The controller includes processors and memory with instructions that when executed by the processors, cause the processors to display the symbology on the display, determine a quality-of-service value based on at least the signal quality value, and modify at least one symbol of the symbology based on the quality-of-service value from a default symbol to a degraded symbol if the quality-of-service value changes from a high-quality value to a low-quality value. The system may also receive sensor signals from other sensors, and use the sensor signal to determine the quality-of-service value.

Abstract: A system and method for determining a position of viewed object is disclosed. The system includes a head/eye tracker configured to sense at least one gaze target in an environment corresponding to a gaze of an eye or head of a first operator and at least one remote object sensor configured to sense an object corresponding to the gaze target. The system further includes at least one controller in communication with the head/eye tracker and the remote object sensor configured to receive gaze target data, transmit commands to the remote object sensor to steer transmission beams toward the viewed object based on the gaze target data, wherein a reflection of the steered transmission beams from the object is received by a receiver, receive reflection data from the receiver, generate object position data based on the reflection data, and transmit the object position data.

Abstract: The system may include at least one a light emitter configured to emit light over a range of wavelengths onto an environment and at least one sensor assembly. The range of wavelengths emitted by the light emitter may be of wavelengths that are above or below wavelengths used by a night vision imaging system. The sensor assembly may include an image sensor and conversion layer. The image sensor may capture images of the environment illuminated by the light emitters and sense light in a given range of wavelengths. The conversion layer may receive the emitted light and convert the emitted light to converted light with a wavelength within the given range of wavelengths associated with the image sensor. The image sensor may be further configured to receive the converted light from the conversion layer, capture an image of the light emitter and output image data associated with the captured image.

Abstract: An imaging system for an aircraft is disclosed. The imaging system comprises one or more image sensors configured to image a surrounding environment of the aircraft, an artifact pattern on an optical component in an optical path of the one or more image sensors, and a controller communicatively coupled to the one or more image sensors. The controller includes one or more processors configured to execute program instructions causing the one or more processors to: receive an image of the surrounding environment of the aircraft from the one or more image sensors, wherein the image includes features of the artifact pattern, extract the features of the artifact pattern from the image, and determine an orientation of the features of the artifact pattern in the image with respect to a calibration artifact pattern.

Abstract: A system and method for head tracking for a head worn display (HWD) includes head-frame and platform-frame IMUs providing high-rate pose data of the wearer's head and a mobile platform. An optical tracker estimates an optical pose of a camera oriented at fiducials based on a comparison of 2D image data captured by the camera (and portraying the fiducials within the image) and the known location of each fiducial in a 3D marker frame. The pIMU also provides low-rate, high-integrity georeferenced pose data of the mobile platform (e.g., in an earth frame). The head tracker provides low-rate, high-integrity absolute head pose solutions (e.g., user head pose in the earth frame) based on the georeferenced pose data and current high-rate head/platform pose data, updating the absolute pose solutions with high-rate updates based on combined extended Kalman filter propagation of the high-rate head and platform pose data with refined optical pose data.

Abstract: A system includes at least one camera implemented in an aircraft, the at least one camera configured to capture images external to the aircraft. The system further includes at least one processor implemented in the aircraft, one or more of the at least one processor communicatively coupled to the at least one camera, the at least one processor configured to: receive the images from the at least one camera; identify visual references in at least one of the images; and output an annunciation instruction configured to cause an annunciation to a pilot that the visual references have been identified by the at least one processor.

Abstract: A system and method for conformal alignment of an aircraft-based head worn display (HWD) system defines far-field features in view of the HWD display, each far-field feature having a 3D truth position in a navigational reference frame. The aircraft GPS/IRS determines pointing vectors to each feature in the aircraft reference frame. The HWD user estimates a nominal head pose for orienting the far-field features in the HWD display, and the HWD system renders the far-field features based on the nominal head pose and pointing vectors. The rendered far-field features are aligned to their real-world 3D truth positions (either manually or with the assistance of witness cameras) and a latched pose of the HWD headtracker system determined based on a successful alignment within accuracy bounds. Based on the latched headtracker pose, the HWD headtracker alignment and the alignment of the HWD display reference frame to the aircraft reference frame are updated.

Abstract: A high-assurance headtracking system for a head worn display (HWD) incorporating ground-truth fiducials is disclosed. In embodiments, the headtracking system includes a camera fixed to a mobile platform (e.g., an aircraft cockpit) and oriented toward a pilot wearing the HWD. Images captured by the camera detect fiducial markers attached to the HWD (and fixed in a device reference frame) and static fiducial markers fixed to the mobile platform, each static fiducial marker including a constellation of individual markers having a known position and orientation in the platform reference frame. The headtracking system include control processors for analyzing the images and identifying therein the device-frame and static fiducial markers. The headtracking system determines a device pose of the HWD relative to the platform reference frame and monitors the device pose for drift or error based on the known positions of the identified static fiducial markers.

Abstract: A system and method for predicting an estimated head pose solution of a head worn display (HWD) at a future time selects or determines, for each HWD rotational axis, a relative angular rate of a head reference frame relative to a body reference frame. By sampling the relative angular rate, a relative angular acceleration is determined. Based on a projection time associated with estimated display latency, angular increment vectors for each rotational axis are determined for relative angular rate and relative angular acceleration. The current head pose solution is adjusted with respect to one or more axes based on the angular increment vectors to project the head pose estimate ahead based on the projection time, the resulting projected head pose solution corresponding to a predicted orientation of the head frame relative to the body frame at the future time.

Abstract: A system includes a head wearable display (HWD) device and a processor. The HWD device includes a waveguide display and an image monitor sensor. The waveguide display includes a waveguide and an optical system, the optical system configured to: receive image data corresponding to an image; and project the image at least through the waveguide to be displayed to a user. The image monitor sensor is optically coupled to the waveguide or the optical system. The image monitor sensor is configured to: capture at least a portion of the image as a monitored image; and output monitored image data. The processor is configured to: receive the monitored image data from the image monitor sensor; and based at least on the monitored image data, determine whether the waveguide display displayed the image correctly.

Abstract: A system and method for performing a self-diagnostic test on an electronically scanned array is disclosed. The system includes an array of emitter antenna and receiver antenna elements, a controller configured to control the modulation of transmitting and received signals, and a monitoring processor configured to receive a signal quality input based on the transmitting and received signal, generate a control profile based on the signal quality input, compare at least one control profile to at least one of a predicted result signal or to control profiles from at least two other sets of control profiles, and determine a faulty set of individually addressable components that includes at least one faulty individually addressable component. The controller is configured to enhance one or more individually addressable components to compensate for the faulty addressable component.

Abstract: A system and method for modifying symbology displayed on a head worn display (HWD) based on a quality-of-service value is disclosed. The system includes a head worn display configured to display symbology, a tracking camera, a controller, and a tracking processing unit configured to receive inputs form the tracking camera and transmit a signal quality value to the controller. The controller includes processors and memory with instructions that when executed by the processors, cause the processors to display the symbology on the display, determine a quality-of-service value based on at least the signal quality value, and modify at least one symbol of the symbology based on the quality-of-service value from a default symbol to a degraded symbol if the quality-of-service value changes from a high-quality value to a low-quality value. The system may also receive sensor signals from other sensors, and use the sensor signal to determine the quality-of-service value.

Abstract: A system includes a vehicle cockpit and a processor. The vehicle cockpit includes imperceptible cockpit reference fiducials and a head wearable display (HWD) device. The imperceptible cockpit reference fiducials are imperceptible to a naked eye of a user in the vehicle cockpit. The HWD device includes a display and an optical sensor configured to: capture images of the imperceptible cockpit reference fiducials; and output optical sensor image data. The processor is configured to: receive the optical sensor image data; and based at least on the optical sensor image data, determine a position and orientation of the head wearable display device. The display is configured to display the images aligned with the field of view of the user based at least on the determined position and the determined orientation of the head wearable display device.

Abstract: An augmented reality system is disclosed that includes an image source configured to generate an image, and a display element configured to display the image. The system further includes an optical relay configured to project the image onto the display element. The system further includes a light sensor configured to quantify an amount of ambient light passing through, the display element. The system further includes a tracker processing unit operatively coupled to the light sensor, the image source, and the optical relay. The tracker processing unit includes a processor, and a memory configured to store instructions executable by the processors. The processors are instructed to receive sensor data from the light sensor, receive the image from the image source, update a pixel characteristic for one or more pixels of the image based on the sensor data, and transmit an updated image to the optical relay.

Abstract: A system is disclosed that display configured to transmit a signal that includes a first light configured as a display image and a second light configured as a reference image. The system also includes a beam splitter, a selectively transmissive mirror and a sensor configured to detect the second light. The beam splitter is configured to receive the signal from the display and reflect the signal to the selectively transmissive mirror. The selectively transmissive mirror is configured to receive the reflected signal, transmit the second light toward a sensor, and reflect the first light, wherein the first light is configured with a first bandwidth, wherein the second light is configured with a second bandwidth. The system further includes a corrector lens configured to receive the first light and transmit the first light to an exit pupil.

Abstract: A system and method for high integrity head pose estimation receives a 2D image including medium-density fiducials and low-density fiducials. The system then detects the medium-density fiducials and compares them with a 3D constellation to form an initial head pose estimate. The initial head pose estimate is then checked for possible alternative correspondences. A selection from a set of candidate initial head pose estimates is made to determine. The selected head pose estimate then has search areas established for the medium-density fiducials and low-density fiducials. The system then determines whether or not the head pose is valid and accurate based on whether or not the fiducials appear in their respective search areas.

Abstract: A system and method for head tracking for a head worn display (HWD) includes head-frame and platform-frame IMUs providing high-rate pose data of the wearer's head and a mobile platform. An optical tracker estimates an optical pose of a camera oriented at fiducials based on a comparison of 2D image data captured by the camera (and portraying the fiducials within the image) and the known location of each fiducial in a 3D marker frame. The pIMU also provides low-rate, high-integrity georeferenced pose data of the mobile platform (e.g., in an earth frame). The head tracker provides low-rate, high-integrity absolute head pose solutions (e.g., user head pose in the earth frame) based on the georeferenced pose data and current high-rate head/platform pose data, updating the absolute pose solutions with high-rate updates based on combined extended Kalman filter propagation of the high-rate head and platform pose data with refined optical pose data.

Abstract: An integrity monitoring system for a first image sensor includes an electronic processor configured to receive sensor data for the provision of image data associated with an environment of the avionic sensor. The electronic processor is configured to monitor the avionic sensor for integrity. The electronic processor is configured to perform at least one of: determining a presence of an optical feature associated with optics of the first image sensor, comparing overlap information derived from the sensor data and other sensor data, comparing characteristics of a digital output stream of the sensor data to expected characteristics, or comparing a first motion derived from the image data and a second motion derived from avionic position equipment.

Abstract: A system is disclosed that display configured to transmit a signal that includes a first light configured as a display image and a second light configured as a reference image. The system also includes a beam splitter, a selectively transmissive mirror and a sensor configured to detect the second light. The beam splitter is configured to receive the signal from the display and reflect the signal to the selectively transmissive mirror. The selectively transmissive mirror is configured to receive the reflected signal, transmit the second light toward a sensor, and reflect the first light, wherein the first light is configured with a first bandwidth, wherein the second light is configured with a second bandwidth. The system further includes a corrector lens configured to receive the first light and transmit the first light to an exit pupil.