Abstract: An onboard aircraft landing system includes one or more event-based cameras disposed at known locations to capture the runway and visible surrounding features such as lights and runway markings. The event-based cameras produce a continuous stream of event data that may be quickly processed to identify both light and dark features contemporaneously, and calculate an aircraft pose relative to the runway based on the identified features and the known locations of the event-based cameras. Composite features are identified via the relative location of individual features corresponding to pixel events.

Abstract: An aircraft-based system for tracking and verifying control inputs from potentially incapacitated pilot stores to memory expected control input sets, e.g., control procedures for controlling an aircraft through segments of a flight plan. When pilot monitors indicate a state of potential incapacitation of a pilot (e.g., fatigue, unresponsiveness, hypoxia), the control input tracking system records any control inputs submitted via the flight deck controls by the incapacitated pilot, reviewing each recorded control input compared to expected control inputs (e.g., for the current flight segment). When a recorded control input sufficiently deviates from the corresponding expected control input (e.g., such that aircraft safety or mission criticality may be affected), the pilot and/or flight crew may receive feedback alerting to the deviant control input (e.g., which may be overridden or which may require confirmation prior to execution).

Abstract: A system includes at least one camera implemented in an aircraft, the at least one camera configured to capture images external to the aircraft. The system further includes at least one processor implemented in the aircraft, one or more of the at least one processor communicatively coupled to the at least one camera, the at least one processor configured to: receive the images from the at least one camera; identify visual references in at least one of the images; and output an annunciation instruction configured to cause an annunciation to a pilot that the visual references have been identified by the at least one processor.

Abstract: A system comprising a head worn display and an image rendering computing device is disclosed. The head worn display comprises one or more image sensors, a head position sensor, and a head position computing device configured to determine a first head pose. The image rendering computing device is configured to render an enhanced image and embed the first head pose in the enhanced image. The head position computing platform is further configured to determine a second head pose, determine a difference between the second head pose and the first head pose, and warp the enhanced image based on the difference between the second head pose and the first head pose to correct for head motion of the user.

Abstract: A system includes a vehicle cockpit and a processor. The vehicle cockpit includes imperceptible cockpit reference fiducials and a head wearable display (HWD) device. The imperceptible cockpit reference fiducials are imperceptible to a naked eye of a user in the vehicle cockpit. The HWD device includes a display and an optical sensor configured to: capture images of the imperceptible cockpit reference fiducials; and output optical sensor image data. The processor is configured to: receive the optical sensor image data; and based at least on the optical sensor image data, determine a position and orientation of the head wearable display device. The display is configured to display the images aligned with the field of view of the user based at least on the determined position and the determined orientation of the head wearable display device.

Abstract: A system and method for high-confidence error overbounding of multiple optical pose solutions receives a set of candidate correspondences between 2D image features captured by an aircraft camera and 3D constellation features including at least one ambiguous correspondence. A candidate estimate of the optical pose of the camera is determined for each of a set of candidate correspondence maps (CMAP), each CMAP resolving the ambiguities differently. Each candidate pose estimate is evaluated for viability and any non-viable estimates eliminated. An individual error bound is determined for each viable candidate pose estimate and CMAP, and based on the set of individual error bounds a multiple-pose containment error bound is determined, bounding with high confidence the set of candidate CMAPs and multiple pose estimates where at least one is correct. The containment error bound may be evaluated for accuracy as required for flight operations performed by aircraft-based instruments and systems.

Abstract: A vision-based navigation system (e.g., for aircraft on approach to a runway) captures via camera 2D images of the runway environment in an image plane. The vision-based navigation system stores a constellation database of runway features and their nominal 3D position information in a constellation plane. Image processors detect within the captured images 2D features potentially corresponding to the constellation features. The vision-based navigation system estimates optical pose of the camera in the constellation plane by aligning the image plane and constellation plane into a common domain, e.g., via orthocorrection of detected image features into the constellation plane or reprojection of constellation features into the image plane. Based on the common-domain plane, the vision-based navigational system generates candidate correspondence maps (CMAP) of constellation features mapped to the image features with high-confidence error bounding, from which optical pose of the camera or aircraft can be estimated.

Abstract: A system and method utilizing an existing boresighted HUD and a camera to determine the transformational and rotational relationships between the camera image and the boresighted HUD image of an aircraft to properly orient the camera for landing operations. Known boresight features are identified in a HUD image and are related to the same boresight features in a camera image, and transformations are applied to align pixels in the camera image to the pixel locations of the boresight features in the HUD image. Object recognition algorithms may be employed to identify the boresight features in the camera image.

Abstract: A system and method are disclosed for design of a suite of multispectral (MS) sensors and processing of enhanced data streams produced by the sensors for autonomous aircraft flight. The onboard suite of MS sensors is specifically configured to sense and use a MS variety of sensor-tuned objects, either strategically placed objects and/or surveyed and sensor significant existing objects to determine a position and verify position accuracy. The received MS sensor data enables an autonomous aircraft object identification and positioning system to correlate MS sensor data output with a-priori information stored onboard to determine and verify position and trajectory of the autonomous aircraft. Once position and trajectory are known, the object identification and positioning system commands the autonomous aircraft flight management system and autopilot control of the autonomous aircraft.

Abstract: An integrity monitoring system for a first image sensor includes an electronic processor configured to receive sensor data for the provision of image data associated with an environment of the avionic sensor. The electronic processor is configured to monitor the avionic sensor for integrity. The electronic processor is configured to perform at least one of: determining a presence of an optical feature associated with optics of the first image sensor, comparing overlap information derived from the sensor data and other sensor data, comparing characteristics of a digital output stream of the sensor data to expected characteristics, or comparing a first motion derived from the image data and a second motion derived from avionic position equipment.

Abstract: A system comprising a head worn display and an image rendering computing device is disclosed. The head worn display comprises one or more image sensors, a head position sensor, and a head position computing device configured to determine a first head pose. The image rendering computing device is configured to render an enhanced image and embed the first head pose in the enhanced image. The head position computing platform is further configured to determine a second head pose, determine a difference between the second head pose and the first head pose, and warp the enhanced image based on the difference between the second head pose and the first head pose to correct for head motion of the user.

Abstract: An onboard aircraft landing system includes one or more event-based cameras disposed at known locations to capture the runway and visible surrounding features such as lights and runway markings. The event-based cameras produce a continuous stream of event data that may be quickly processed to identify both light and dark features contemporaneously, and calculate an aircraft pose relative to the runway based on the identified features and the known locations of the event-based cameras. Composite features are identified via the relative location of individual features corresponding to pixel events.

Abstract: A system and method are provided for automatically adjusting exposure to maintain color fidelity where a point source puts a small cluster of pixels in saturation. An exposure level is iteratively reduced until pixels corresponding only one color in a Bayer filter remain above a threshold while all other colors are at or below the threshold. The cluster of pixels is isolated for analysis such that only neighboring pixels are considered while reducing the exposure.

Abstract: A system and method are provided for automatically adjusting exposure to maintain color fidelity where a point source puts a small cluster of pixels in saturation. An exposure level is iteratively reduced until pixels corresponding only one color in a Bayer filter remain above a threshold while all other colors are at or below the threshold. The cluster of pixels is isolated for analysis such that only neighboring pixels are considered while reducing the exposure.

Abstract: A system and method are disclosed for design of a suite of multispectral (MS) sensors and processing of enhanced data streams produced by the sensors for autonomous aircraft flight. The suite of MS sensors is specifically configured to produce data streams for processing by an autonomous aircraft object identification and positioning system processor. Multiple, diverse MS sensors image naturally occurring, or artificial features (towers buildings etc.) and produce data streams containing details which are routinely processed by the object identification and positioning system yet would be unrecognizable to a human pilot. The object identification and positioning system correlates MS sensor output with a-priori information stored onboard to determine position and trajectory of the autonomous aircraft. Once position and trajectory are known, the object identification and positioning system sends the data to the autonomous aircraft flight management system for autopilot control of the autonomous aircraft.

Abstract: An integrity monitoring system for a first image sensor includes an electronic processor configured to receive sensor data for the provision of image data associated with an environment of the avionic sensor. The electronic processor is configured to monitor the avionic sensor for integrity. The electronic processor is configured to perform at least one of: determining a presence of an optical feature associated with optics of the first image sensor, comparing overlap information derived from the sensor data and other sensor data, comparing characteristics of a digital output stream of the sensor data to expected characteristics, or comparing a first motion derived from the image data and a second motion derived from avionic position equipment.

Abstract: A system and method operate to leverage high assurance positioning systems to determine a hybrid deviation from an expected position, altitude, and path to generate a lateral and vertical signal indicating deviation from a desired position. As a filtered or alternative to conventional precision landing systems, the hybrid positioning system receives sensor data from a plurality of sensors and compares the received sensor data to known values to determine a deviation of position and trajectory. Extending the capability of existing auto-land systems, relative-to-runway sensors and/or filtered high assurance hybrid solutions are employed as an alternative to conventional precision landing systems. The sensors provide position data that is compared to an expected position and path to provide deviations in lateral, vertical, and speed augmenting traditional RF-based systems.

Abstract: An integrity monitoring system for a first image sensor includes an electronic processor configured to receive sensor data for the provision of image data associated with an environment of the avionic sensor. The electronic processor is configured to monitor the avionic sensor for integrity. The electronic processor is configured to perform at least one of: determining a presence of an optical feature associated with optics of the first image sensor, comparing overlap information derived from the sensor data and other sensor data, comparing characteristics of a digital output stream of the sensor data to expected characteristics, or comparing a first motion derived from the image data and a second motion derived from avionic position equipment.

Abstract: A computer system with one or more cameras monitors the head poses produced by a head-tracking system and produces a pass/fail indication that the head poses are reliable in a high assurance environment. The system may identify individual fiducials and independently verifying that they conform to the computed pose; and/or compute a location of fiducials as would be observed by a single verification camera and compare the computed locations to actual observed locations. The verification camera may be part of a second head-tracking system, utilized while not operating in its ordinary capacity. The computer system may also calibrate the head-tracking system to account for temperature and pressure fluctuations and perform image stabilization before the image frames are passed to the head-tracking system for processing by observing separate fiducials in fixed locations relative to the camera.

Abstract: A system and method are disclosed for design of a suite of multispectral (MS) sensors and processing of enhanced data streams produced by the sensors for autonomous aircraft flight. The onboard suite of MS sensors is specifically configured to sense and use a MS variety of sensor-tuned objects, either strategically placed objects and/or surveyed and sensor significant existing objects to determine a position and verify position accuracy. The received MS sensor data enables an autonomous aircraft object identification and positioning system to correlate MS sensor data output with a-priori information stored onboard to determine and verify position and trajectory of the autonomous aircraft. Once position and trajectory are known, the object identification and positioning system commands the autonomous aircraft flight management system and autopilot control of the autonomous aircraft.