Vision system for automatically aligning a passenger boarding bridge with a doorway of an aircraft and method therefor
A vision system for use with an automated control system of a passenger boarding bridge includes an inclinometer for determining tilt data relating to deviation of the aircraft-engaging end of the passenger boarding bridge relative a horizontal reference plane. The system also includes an imager disposed near the aircraft-engaging end of the passenger boarding bridge, for capturing image data relating to a portion of the aircraft proximate an expected stopping location of the doorway. A memory element having template image data stored retrievably therein is also provided. The template image data relates to at least a template image including a feature that is indicative of the location of the doorway of the aircraft. The vision system further includes an image data processor for determining alignment data for use in aligning the aircraft-engaging end of the passenger boarding bridge with the doorway of the aircraft. The alignment data being determined based upon the tilt data, the image data, and the template image data.
Latest DEW Engineering and Development Limited Patents:
- Conveyor system for loading and unloading aircraft baggage
- Door seal assembly for armored vehicles
- Automated passenger boarding bridge alignment system and method with manual identification of aircraft doorway position
- Passenger boarding bridge stabilizing apparatus and method therefor
- System and method for guiding an aircraft to a stopping position
The instant invention relates generally to passenger boarding bridges, and more particularly to a vision system and method for automated passenger boarding bridges.
BACKGROUNDIn order to make aircraft passengers comfortable, and in order to transport them between an airport terminal building and an aircraft in such a way that they are protected from the weather and from other environmental influences, passenger boarding bridges are used which are telescopically extensible and the height of which is adjustable. For instance, an apron drive bridge includes a plurality of adjustable modules, including: a rotunda, a telescopic tunnel, a bubble section, a cab, and elevating columns with wheel carriage. Other common types of passenger boarding bridges include radial drive bridges and over-the-wing (OTW) bridges. These types of passenger boarding bridges are adjustable, for instance to compensate for different sized aircraft and to compensate for imprecise parking of aircraft at an airport terminal.
Historically, the procedure for aligning the passenger boarding bridge with the doorway of an aircraft has been a time consuming and labor intensive operation. First, the pilot taxis the aircraft along a lead-in line to a final parking position within a gate area. Typically, the lead-in line is a physical marker that is painted onto the tarmac, and is used for guiding the aircraft along a predetermined path to a final parking position. Additional markings in the form of stop lines, one for each type of aircraft, are provided at predetermined positions along the lead-in line. Thus, when the nose gear of a particular type of aircraft stops precisely at the stop line for that type of aircraft, then the aircraft is known to be at its final parking position. Of course, the pilot's view of the tarmac surface from the cockpit of an aircraft is limited. This is particularly true for larger aircraft, such as for instance a Boeing 747-X00. Typically, the pilot has relied upon instructions that are provided by a human ground marshal together with up to two “wing walkers” to follow the lead-in line. Optionally, stop bars are located on a pole that is fixedly mounted to the ground surface, including appropriate stop bars for each type of aircraft using the gate. Alternatively, a tractor or tug is used to tow the aircraft along the lead-in line to its final parking position.
More recently, sophisticated Visual Docking Guidance Systems have been developed to perform the function of the human ground marshal or tug operator. In particular, a Visual Docking Guidance System (VDGS) senses the aircraft as it approaches the final parking position and provides instructions to the pilot via an electronic display device. The electronic display device is mounted at a location that makes it highly visible to the pilot when viewed from the cockpit of an aircraft. Typically, the instructions include a combination of alphanumeric characters and symbols, which the pilot uses to guide the aircraft precisely to the final parking position for the particular type of aircraft. The high capital cost of the VDGS system is offset by reduced labor costs and the efficiency that results from stopping the aircraft more precisely than is possible under the guidance of a human ground marshal.
Of course, even when the aircraft is stopped precisely at the final parking position for that type of aircraft, still there is the matter of moving the passenger boarding bridge into an aligned relationship with a doorway of the parked aircraft. In the case of an apron drive bridge this may involve extending the bridge by 10 to 20 meters or more from a stowed position. Unfortunately, driving the bridge over such a long distance is time consuming because often the rate at which the bridge is moved is limited so as to reduce the risk of colliding with ground service vehicles or personnel, and to avoid causing serious damage to the aircraft in the event of a collision therewith. Manual, semi-automated and automated bridge alignment systems are known for moving the passenger boarding bridge relative to the parked aircraft.
A manual bridge alignment system requires that a human operator is present to perform the alignment operation each time an aircraft arrives. Delays occur when the human operator is not standing-by to perform the alignment operation as soon as the aircraft comes to a stop. In addition, human operators are prone to errors that may result in the passenger boarding bridge being driven into the aircraft or into a piece of ground service equipment. Such collisions involving the passenger boarding bridge are costly and also result in delays. In order to avoid causing a collision, human operators tend to err on the side of caution and drive the passenger boarding bridge slowly and cautiously.
Semi-automated bridge alignment systems are also known, whereby the bridge is moved rapidly to a preset position under the control of a computer. WO 96/08411, filed Sep. 14, 1995 in the name of Anderberg, describes a semi-automated system for controlling the movement of a passenger boarding bridge. When an aircraft has landed, a central computer transmits information relating to the type of the approaching aircraft to a local computer of the passenger boarding bridge. The local computer accesses a database and retrieves information relating to the positions of the doors for the type of aircraft that has landed, as well as information relating to the expected final parking position for the type of aircraft. The local computer uses the retrieved information to determine an absolute position of the door to which the bridge is to be aligned. Accordingly, the passenger boarding bridge is moved under computer control to a position that is close to the determined position of the door, for example within 2-10 meters. Optionally, the bridge is preset to this position before the aircraft stops moving.
WO 01/34467, filed Nov. 8, 2000 also in the name of Anderberg, teaches that the above system is reliable only for movement to a position that is close to the parked aircraft. Thus, the bridge has to be operated manually during the remaining 2-10 meters of its movement. The WO 01/34467 reference also teaches an improvement to the above system, in which electromagnetic sensors are disposed along the outboard end of the passenger boarding bridge for transmitting a set of electromagnetic pulses in different directions and for detecting electromagnetic pulses after reflection from an aircraft. Based on the elapsed time between transmitting and detecting the electromagnetic pulses in different directions, a profile of distance as a function of direction is obtained. From the measured distance versus direction profile and the information that is stored in the computer, it is then possible to maneuver the bridge the rest of the way from the preset position to the door of the parked aircraft. Unfortunately, when the aircraft fails to stop at the expected final parking position, the preset position will be misaligned with the actual position of the aircraft door, and human intervention will be required in order to complete the alignment operation.
Other automated systems have been proposed, such as for instance an automated passenger boarding bridge that uses video cameras in the control of the bridge as described by Schoenberger et al. in U.S. Pat. No. 5,226,204. The system uses video cameras to capture images of an aircraft with which the bridge is to be aligned, which images are provided to a computer for image processing. An object of the image processing is to locate doors along the lateral surface of the aircraft facing the passenger boarding bridge. The bridge is then moved automatically along a direction toward a predetermined door of the parked aircraft. Unfortunately, the system that is described in U.S. Pat. No. 5,226,204 suffers from several disadvantages. For instance, a sophisticated image processing system is required in order to locate a doorway along the side of an aircraft from a distance of up to 15 meters or more. In addition, the “blob” approach and edge detection image processing techniques that are mentioned in the reference are computationally expensive and inherently slow. It is a further disadvantage that factors such as the weather, ambient lighting conditions, markings on the aircraft and the presence of intervening ground support vehicles may also become very significant over such large distances. Furthermore, the bridge is still required to move a significant distance after the aircraft has come to a stop, which increases the time that is needed to complete the alignment and poses a hazard to ground service vehicles and personnel.
Yet another type of system is disclosed in published United States patent application 2005/0198750 A1, filed Feb. 26, 2003 in the name of Spencer et al. In particular, reflective targets are affixed to the outside surface of an aircraft around the doorway to which the passenger boarding bridge is to be aligned. A plurality of cameras disposed aboard the passenger boarding bridge is used to image the targets during the alignment procedure, and provides image data to a “computer means” for processing thereby. While the use of reflective targets for identifying the doorway of the aircraft is advantageous in that it simplifies image processing, never-the-less airlines are reluctant to apply targets to their aircraft and furthermore regulatory approval may be necessary in order to do so. In addition, the system is likely to fail if the targets become obscured due to dirt, scuffs, tearing or the build up of snow, etc. Another disadvantage of the system that is described by Spencer et al. is that the targets are imaged continuously as the bridge moves into alignment with the doorway of the aircraft. Continuous imaging, and the associated processing of image data, is necessary in order to update the alignment data so as to compensate for initial low accuracy of the system as well as unexpected movement of the bridge during alignment, such as for instance movement resulting from uneven ground surface adjacent the passenger boarding bridge. Accordingly, the bridge is required to move a significant distance at relatively slow speed, which may unacceptably increase alignment time.
Accordingly, there is a long-standing and unfulfilled need for a bridge alignment system that is capable of safely and reliably aligning a passenger boarding bridge with an aircraft, absent intervention by a human operator. It would be advantageous to provide a system that overcomes at least some of the above-mentioned disadvantages of the prior art.
SUMMARY OF EMBODIMENTS OF THE INVENTIONIn accordance with an aspect of the instant invention there is provided a vision system for use with an automated control system of a passenger boarding bridge, the automated control system for aligning an aircraft-engaging end of the passenger boarding bridge with a doorway of an aircraft that is stopped within a defined parking space adjacent to the passenger boarding bridge, the vision system comprising: an imager disposed proximate the aircraft-engaging end of the passenger boarding bridge for capturing image data relating to a portion of the aircraft that is proximate an expected stopping location of the doorway; an inclinometer for determining tilt data relating to a sensed deviation of the aircraft-engaging end of the passenger boarding bridge relative a horizontal reference plane; a distance sensor disposed proximate the aircraft-engaging end of the passenger boarding bridge for sensing a distance to the portion of the aircraft that is proximate the expected stopping location of the doorway; a memory element having template image data stored retrievably therein, the template image data comprising at least a template image including a feature that is indicative of the location of the doorway of the aircraft; and, an image data processor for transforming the template image data based on the tilt data and the sensed distance, for determining a correlation between the feature that is indicative of the location of the doorway of the aircraft in the transformed template image data and a corresponding feature in the captured image data, and for determining alignment data for use in aligning the aircraft-engaging end of the passenger boarding bridge with the doorway of the aircraft based on the determined correlation.
In accordance with another aspect of the instant invention there is provided a method for aligning an aircraft-engaging end of a passenger boarding bridge with a doorway of an aircraft, the aircraft stopped within a defined parking space adjacent to the passenger boarding bridge, the method comprising: positioning the passenger boarding bridge relative to the aircraft, such that the aircraft-engaging end of the passenger boarding bridge is adjacent to an expected stopping location of the doorway of the aircraft; capturing image data using an imager that is disposed proximate the aircraft-engaging end of the passenger boarding bridge, the image data including features that are located within a portion of the aircraft that is proximate the expected stopping location of the doorway; sensing orientation data including tilt data, the orientation data relating to a current orientation of the aircraft-engaging end of the passenger boarding bridge when the aircraft-engaging end of the passenger boarding bridge is positioned adjacent to the expected stopping location of the doorway of the aircraft; retrieving template image data relating to the doorway of the aircraft; transforming the template image data based on the sensed orientation data; comparing the captured image data with the transformed template image data, to determine instruction data relating to a horizontal movement and a vertical movement for aligning the aircraft-engaging end of the passenger boarding bridge with the doorway of the aircraft; providing the instruction data to an automated control system of the passenger boarding bridge; and, under the control of the automated control system, performing the horizontal movement and the vertical movement for aligning the aircraft-engaging end of the passenger boarding bridge with the doorway of the aircraft.
In accordance with another aspect of the instant invention there is provided a method for aligning an aircraft-engaging end of a passenger boarding bridge with a doorway of an aircraft, comprising: using a visual docking guidance system (VDGS), guiding the aircraft to a predetermined stopping position adjacent to the passenger boarding bridge; positioning the aircraft-engaging end of the passenger boarding bridge at a known position for capturing an image of a portion of the aircraft that is proximate an expected stopping location of the doorway; using an imager disposed proximate the aircraft-engaging end of the passenger boarding bridge, capturing an image of the portion of the aircraft that is proximate the expected stopping location of the doorway; sensing orientation data relating to tilt of the aircraft-engaging end of the passenger boarding bridge relative a horizontal reference plane, and relating to distance between the aircraft-engaging end of the passenger boarding bridge and the portion of the aircraft that is proximate the expected stopping location of the doorway; retrieving a template image from a memory element, the template image relating to the doorway of the aircraft; transforming the template image based on the sensed orientation data; based on a comparison between the transformed template image and the captured image, determining instruction data for moving the aircraft-engaging end of the passenger boarding bridge into an aligned condition with the doorway of the aircraft from the known position; and, under the control of an automated control system of the passenger boarding bridge, moving the aircraft-engaging end of the passenger boarding bridge in accordance with the instruction data, so as to align the aircraft-engaging end of the passenger boarding bridge with the doorway of the aircraft.
In accordance with another aspect of the instant invention there is provided a method for aligning an aircraft-engaging end of a passenger boarding bridge with a doorway of an aircraft, the aircraft stopped within a defined parking space adjacent to the passenger boarding. bridge, the method comprising: moving the aircraft-engaging end of the passenger boarding bridge to a known position adjacent to an expected stopping location of the doorway of the aircraft; capturing first image data using a first imager that is disposed proximate the aircraft-engaging end of the passenger boarding bridge, the first image data relating to a first portion of the aircraft that is proximate the expected stopping location of the doorway; capturing second image data using a second imager disposed proximate the aircraft-engaging end of the passenger boarding bridge, the second image data relating to a second portion of the aircraft that is proximate the expected stopping location of the doorway; sensing orientation data including tilt data, the orientation data relating to an orientation of the aircraft-engaging end of the passenger boarding bridge after moving to the known position; retrieving template image data relating to the doorway of the aircraft; transforming the template image data based on the sensed orientation data; comparing separately the captured first image data and the captured second image data with the transformed template image data, so as to determine independently first alignment data and second alignment data, respectively; selecting one of the first alignment data and the second alignment data based on a predefined selection criterion; and, aligning the aircraft-engaging end of the passenger boarding bridge with the doorway of the aircraft based upon the selected one of the first alignment data and the second alignment data.
Exemplary embodiments of the invention will now be described in conjunction with the following drawings, in which similar reference numbers designate similar items:
The following description is presented to enable a person skilled in the art to make and use the invention, and is provided in the context of a particular application and its requirements. Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and the scope of the invention. Thus, the present invention is not intended to be limited to the embodiments disclosed, but is to be accorded the widest scope consistent with the principles and features disclosed herein.
Referring to
Referring now to
The camera enclosure 114 also houses an inclinometer 130, such as for instance a ceramic electrolytic tilt sensor, which is offset 20° upwards relative to the imager 126. Accordingly, the inclinometer 130 is approximately parallel to the floor of the cabin 108. A distance sensor such as for instance a laser range finder (LRF) 132 also is housed within the camera enclosure 114. In the specific example that is shown in
Referring now to
Operation of the vision system is described in the following paragraphs, with specific reference being made to the various components that are shown in
Separately, data that is indicative of the type and sub-type of the aircraft 100 is provided to the controller 112, which then provides a control signal for moving the passenger boarding bridge 102 into a photo position for that particular type and sub-type of aircraft. For instance, the data is provided manually via an interface having a different button for each type and sub-type of aircraft or having a series of alphanumeric buttons. Optionally, a sensor is used to sense the type and sub-type of the aircraft 100 and sensor data is provided to the controller 112. Further optionally, the type and sub-type data is provided from the VDGS 124 or from the FIDS of the airport. The photo position is defined such that the camera enclosure 114 is approximately aligned with the doorway of the aircraft if the aircraft stops precisely at the parking space that is adjacent to the passenger boarding bridge 102. Since different aircraft types and sub-types have different fuselage configurations, that is to say, the doorways are located at different positions and heights along the side of the aircraft, most different aircraft types and sub-types are expected to require a unique photo position. The passenger boarding bridge optionally is moved to the photo position either before or after the aircraft 100 has come to a stop at the parking space.
Once the passenger boarding bridge 102 stops at the photo position and the aircraft 100 stops at the parking space, the LRF 132 is used to sense the distance to the aircraft, which is done under the control of controller 112. The LRF 132 is disposed above or adjacent to the imager 126, such that the measured distance corresponds to the actual distance from the camera enclosure 114 to the aircraft 100. Controller 112 compares the distance value that is returned by LRF 132 to a range of expected values. If the distance value falls within the range of expected values then the distance value is considered to be valid. However, in some instances the LRF 132 does not return a valid distance value. For instance, if the LRF 132 coincidentally is directed toward a window of the aircraft or toward an area of blue or other dark colored paint, then the distance value that is returned by LRF 132 will fall outside the range of expected values and is not likely to be valid. In such an instance manual alignment of the passenger boarding bridge may be necessary. Optionally, the controller 112 moves the passenger boarding bridge 102 into a back-up photo position, and LRF 132 is used in a second attempt to sense the distance to the aircraft 100. For instance, the bridge is moved from the photo position to the back-up photo position by reversing a portion of the movement that brought the bridge into the original photo position. Since the bridge is merely reversing its original course in order to arrive at the back-up photo position, there is very little risk of causing damage to the aircraft or ground service equipment. If a valid distance value is obtained from the back-up photo position, then the automated alignment operation proceeds absent human intervention.
The controller 112 provides distance value data to the image data processor 118, as well as data relating to the type and sub-type of the aircraft. An image is then captured using the imager 126, and image data relating to the captured image is provided to the image data processor 118. In addition, tilt data relating to the pitch and roll of the cabin 108 is measured using the inclinometer 130 and is also provided to the image data processor 118. The image data processor 118 then processes the image data relating to the captured image, making use of the distance data and the tilt data.
The image data processing algorithm is based upon a comparison of features that are contained within the captured image against a small database of template images stored within memory element 120 for each different type and sub-type of aircraft. The database includes a plurality of template images of the outline of the doorway, and shows distinctive features such as the handle of the door, the window of the door, the doorway base plate, etc. In particular, each image of the plurality of template images is obtained from a known optimal viewing position. Optionally, paint scheme features are included in at least some images of the plurality of template images.
Prior to comparison, the template images are transformed using the distance data that is provided by the LRF 132, and using tilt data that is provided by inclinometer 130, respectively. The transformed template images are then compared to the captured image, and horizontal and vertical adjustment values are determined. For instance, it is determined by how much the passenger boarding bridge would have to move in the horizontal and vertical directions to cause the features in the real world images to overlap with the features in the transformed template images. In addition, a rotational movement of the cabin 108 is determined. The rotational movement ensures that the aircraft-engaging end 110 of the cabin 108 is properly aligned with the aircraft fuselage, such that large gaps do not exist therebetween and pose a hazard to passengers or crew. The horizontal and vertical adjustment values may be determined a plurality of times, each time using a different feature of the aircraft to make the determination.
After image data processing is complete, the image data processor 118 returns two values to the controller 112 for specifying the x-axis (horizontal) and y-axis (vertical) adjustment to the passenger boarding bridge 100 that is necessary for aligning the aircraft-engaging end 110 of the cabin 108 precisely with the doorway 122 of aircraft 100. Of course, a value relating to the rotational movement of the cabin 108 also is provided to the controller 112. The controller 112 receives the two values from the image data processor 118, and retrieves a bridge specific offset value from memory element 120. For instance, a bridge specific offset value is stored within memory element 120 for each different photo position and back-up photo position. The bridge specific offset value is added to the y-axis adjustment value, so as to correct for slope or unevenness of the apron surface below the passenger boarding bridge 102. Based upon the corrected two values, the controller 112 determines control signals for moving the passenger boarding bridge 102, and provides the control signals to mechanisms of the automated bridge alignment system for adjusting the height and the angular orientation of the passageway 104. A control signal is also provided for rotationally adjusting the cabin 108. Thereafter, the length of the passageway 104 is extended in order to complete the alignment operation with the doorway 122.
As discussed supra, optionally the imager 126 is provided in the form of a digital camera that is capable of imaging near IR and/or UV. Such an imager is capable of imaging heat signatures around the door or windows of the aircraft. For instance, the IR signature of the window in the door is different than that of the aluminum skin of the aircraft fuselage. Furthermore, use of IR and/or UV imagers may make the paint scheme of the aircraft, which varies considerably from airline to airline, less of an issue.
Referring now to
The camera enclosure 114 also houses an inclinometer 130, such as for instance a ceramic electrolytic tilt sensor, which is offset 20° upwards relative to the imagers 126 and 134. Accordingly, the inclinometer 130 is approximately parallel to the floor of the cabin 108. A distance sensor such as for instance a laser range finder (LRF) 132 also is housed within the camera enclosure 114. In the specific example that is shown in
Referring now to
Operation of the vision system is described in the following paragraphs, with specific reference being made to the various components that are shown in
Separately, data that is indicative of the type and sub-type of the aircraft 100 is provided to the controller 112, which then provides a control signal for moving the passenger boarding bridge 102 into a photo position for that particular type and sub-type of aircraft. For instance, the data is provided manually via an interface having a different button for each type and sub-type of aircraft or having a series of alphanumeric buttons. Optionally, a sensor is used to sense the type and sub-type of the aircraft 100 and sensor data is provided to the controller 112. Further optionally, the type and sub-type data is provided from the VDGS or from the FIDS of the airport. The photo position is defined such that the camera enclosure 114 is approximately aligned with the doorway of the aircraft if the aircraft stops precisely at the parking space that is adjacent to the passenger boarding bridge 102. Since different aircraft types and sub-types have different fuselage configurations, that is to say, the doorways are located at different positions and heights along the side of the aircraft, most different aircraft types and sub-types will require a unique photo position. The passenger boarding bridge optionally is moved to the photo position either before or after the aircraft has come to a stop at the parking space.
Once the passenger boarding bridge 102 stops at the photo position and the aircraft 100 stops at the parking space, the LRFs 132 and 116 are used to sense the distance to the aircraft 100, which is done under the control of controller 112. The LRF 132 is disposed above or adjacent to the imagers 126 and 134, such that the measured distance corresponds to the actual distance from the camera enclosure 114 to the aircraft 100. Controller 112 compares the distance value that is returned by LRF 132 to a range of expected values. If the distance value falls within the range of expected values then the distance value is considered to be valid. However, in some instances the LRF 132 does not return a valid distance value. For instance, if the LRF 132 coincidentally is directed toward a window of the aircraft or toward an area of blue or other dark colored paint, then the distance value that is returned by LRF 132 will fall outside the range of expected values and is not likely to be valid. In such an instance the controller 112 compares the distance value that is measured using the second LRF 116 to a second range of expected values. If the distance value that is measured by the second LRF 116 falls within the second range of expected values, then it is considered to be valid for use during alignment. Since the second LRF 116 is not disposed above or adjacent to the imagers 126 and 134, it is necessary to scale the distance that is returned by the second LRF 116 in order to determine the actual distance to the aircraft relative to the camera enclosure 114. If neither the first LRF 132 nor the second LRF 116 returns a valid distance value, then the automated alignment operation is aborted and a human operator is paged to complete the alignment manually. Optionally, the bridge is moved into a back-up photo position for the determined type and sub-type of the aircraft, and fresh distance values are measured using the first LRF 132 and the second LRF 116. For instance, the bridge is moved from the photo position to the back-up photo position by reversing a portion of the movement that brought the bridge into the photo position. Since the bridge is merely reversing its original course in order to arrive at the back-up photo position, there is very little risk of causing damage to the aircraft or ground service equipment. If a valid distance value is obtained from the back-up photo position, then the automated alignment operation proceeds absent human intervention.
The controller 112 provides distance value data to the image data processor 118, as well as data relating to the type and sub-type of the aircraft. An image is then captured using each of the two imagers 126 and 134, and image data relating to each captured image is provided to the image data processor 118. In addition, tilt data relating to the pitch and roll of the cabin 108 is measured using the inclinometer 130 and is also provided to the image data processor 118. The image data processor then processes separately the image data relating to each of the two captured images, making use of the distance data and the tilt data.
More specifically, the image data processing algorithm is based upon a comparison of features that are contained within the captured images against a small database of template images stored within memory element 120 for each different type and sub-type of aircraft. The database includes a plurality of template images of the outline of the doorway, and shows distinctive features such as the handle of the door, the window of the door, the doorway base plate, etc. Optionally, paint scheme features are included in at least some of the plurality of template images. Each image of the plurality of template images is obtained from a known optimal viewing position.
Prior to comparison, the template images are transformed using distance data provided by one of the LRFs 132 and 1 16, and using tilt data provided by inclinometer 130, respectively. The transformed template images are then compared separately to the two captured images, and horizontal and vertical adjustment values are determined. For instance, it is determined by how much the passenger boarding bridge would have to move in the horizontal and vertical directions to cause the features in the real world images to overlap with the features in the transformed template images. In addition, a rotational movement of the cabin 108 is determined. The rotational movement ensures that the aircraft-engaging end 110 of the cabin 108 is properly aligned with the aircraft fuselage, such that large gaps do not exist therebetween and pose a hazard to passengers or crew. The horizontal and vertical adjustment values may be determined a plurality of times for each captured image, each time using a different feature of the aircraft to make the determination. The determination of which one of the two captured images yields the best values may then be determined statistically.
After image data processing is complete, the image data processor 118 returns two values to the controller 112 for specifying the x-axis (horizontal) and y-axis (vertical) adjustment to the passenger boarding bridge that is necessary for aligning the aircraft-engaging end 110 of the cabin 108 precisely with the doorway 122 of aircraft 100. Of course, a value relating to the rotational movement of the cabin 108 also is provided to the controller 112. The controller 112 receives the two values from the image data processor 118, and retrieves a bridge specific offset value from memory element 120. For instance, a bridge specific offset value is stored within memory element 120 for each different photo position. The bridge specific offset value is added to the y-axis value, and corrects for slope or unevenness of the apron surface below the passenger boarding bridge 102. Based upon the corrected two values, the controller 112 determines control signals for moving the passenger boarding bridge 102, and provides the control signals to mechanisms of the automated bridge alignment system for adjusting the height and the angular orientation of the passageway 104. A control signal is also provided for rotationally adjusting the cabin 108. Thereafter, the length of the passageway 104 is extended in order to complete the alignment operation with the doorway 122.
Although both images are processed fully, the two values that are returned to controller 112 are based upon only one or the other of the two captured images. If neither one of the two captured images is determined to be suitable for use then an error message is generated, and bridge alignment is performed in a manual fashion. This may occur, for instance, when the aircraft is improperly parked and therefore neither image contains features that are indicative of the doorway 122. Optionally, a confidence score is associated with the values that are obtained using each of the two captured images, and those values having the highest confidence score are used for aligning the passenger boarding bridge 102.
As discussed supra, at least one of the imagers 126 and 134 optionally is provided in the form of a digital camera that is capable of imaging near IR and/or UV. Such an imager is capable of imaging heat signatures around the door or windows of the aircraft. For instance, the IR signature of the window in the door is different than that of the aluminum skin of the aircraft fuselage. Furthermore, use of IR and/or UV imagers may make the paint scheme of the aircraft, which varies considerably from airline to airline, less of an issue.
Referring now to
Referring now to
Referring now to
At least two operating modes may be envisaged for those embodiments of the instant invention that include only a single imager 126. In a first operating mode the single imager 126 comprises a wide-angle lens that is capable of imaging a portion of an aircraft that is sufficiently large to reliably capture features that are useful for locating the doorway. It is optional but advantageous to obtain distance values using two separate LRFs when operating in the first operating mode. In a second operating mode the bridge is moved to a first photo position and the single imager 126 is used to capture a first image, then the bridge is moved to a second photo position and the single imager 126 is used to capture a second image. Since the bridge is necessarily moved from one photo position to another photo position when operating in the second operating mode, reliable operation may be achieved even using only a single LRF.
Numerous other embodiments may be envisaged without departing from the spirit and scope of the invention.
Claims
1. A vision system for use with an automated control system of a passenger boarding bridge, the automated control system for aligning an aircraft-engaging end of the passenger boarding bridge with a doorway of an aircraft that is stopped within a defined parking space adjacent to the passenger boarding bridge, the vision system comprising:
- an imager disposed proximate the aircraft-engaging end of the passenger boarding bridge for capturing image data relating to a portion of the aircraft that is proximate an expected stopping location of the doorway;
- an inclinometer for determining tilt data relating to a sensed deviation of the aircraft-engaging end of the passenger boarding bridge relative a horizontal reference plane;
- a distance sensor disposed proximate the aircraft-engaging end of the passenger boarding bridge for sensing a distance to the portion of the aircraft that is proximate the expected stopping location of the doorway;
- a memory element having template image data stored retrievably therein, the template image data comprising at least a template image including a feature that is indicative of the location of the doorway of the aircraft; and,
- an image data processor for transforming the template image data based on the tilt data and the sensed distance, for determining a correlation between the feature that is indicative of the location of the doorway of the aircraft in the transformed template image data and a corresponding feature in the captured image data, and for determining alignment data for use in aligning the aircraft-engaging end of the passenger boarding bridge with the doorway of the aircraft based on the determined correlation.
2. A vision system according to claim 1, wherein the imager comprises a digital camera.
3. A vision system according to claim 1, wherein the imager comprises a plurality of digital cameras including a first digital camera having a first main optical axis and a second digital camera having a second main optical axis, the first main optical axis being angularly separated from the second main optical axis in the horizontal plane.
4. A vision system according to claim 3, wherein the first main optical axis and the second main optical axis define an acute angle therebetween.
5. A vision system according to claim 4, wherein the acute angle is between about 5° and about 25°.
6. A vision system according to claim 3, wherein the first main optical axis is substantially perpendicular to the portion of the aircraft when the aircraft is stopped within the defined parking space.
7. A vision system according to claim 1, wherein the imager supports imaging based on the ultraviolet (UV) region of the electromagnetic spectrum.
8. A vision system according to claim 1, wherein the imager supports imaging based on the infrared (IR) region of the electromagnetic spectrum.
9. A vision system according to claim 1, wherein the distance sensor comprises a laser range finder.
10. A vision system according to claim 1, wherein the distance sensor comprises a plurality of laser range finders that are separated spatially one from another and that are mounted separately proximate the aircraft-engaging end of the passenger boarding bridge.
11. A vision system according to claim 1, comprising a distance-measuring device disposed at a reference location that is remote from the aircraft-engaging end of the passenger boarding bridge and from the aircraft.
12. A vision system according to claim 11, wherein the distance-measuring device is a laser range finder.
13. A vision system according to claim 12, wherein the laser range finder is an element of a visual docking guidance system (VDGS) that is associated with the defined parking space adjacent to the passenger boarding bridge, and that is in communication with the automated control system.
14. A method for aligning an aircraft-engaging end of a passenger boarding bridge with a doorway of an aircraft, the aircraft stopped within a defined parking space adjacent to the passenger boarding bridge, the method comprising:
- positioning the passenger boarding bridge relative to the aircraft, such that the aircraft-engaging end of the passenger boarding bridge is adjacent to an expected stopping location of the doorway of the aircraft;
- capturing image data using an imager that is disposed proximate the aircraft-engaging end of the passenger boarding bridge, the image data including features that are located within a portion of the aircraft that is proximate the expected stopping location of the doorway;
- sensing orientation data including tilt data, the orientation data relating to a current orientation of the aircraft-engaging end of the passenger boarding bridge when the aircraft-engaging end of the passenger boarding bridge is positioned adjacent to the expected stopping location of the doorway of the aircraft;
- retrieving template image data relating to the doorway of the aircraft;
- transforming the template image data based on the sensed orientation data;
- comparing the captured image data with the transformed template image data, to determine instruction data relating to a horizontal movement and a vertical movement for aligning the aircraft-engaging end of the passenger boarding bridge with the doorway of the aircraft;
- providing the instruction data to an automated control system of the passenger boarding bridge; and,
- under the control of the automated control system, performing the horizontal movement and the vertical movement for aligning the aircraft-engaging end of the passenger boarding bridge with the doorway of the aircraft.
15. A method according to claim 14, comprising providing the imager with a wide-angle lens.
16. A method according to claim 14, wherein sensing orientation data including tilt data comprises using an inclinometer disposed proximate the aircraft-engaging end of the passenger boarding bridge for sensing tilt of the aircraft-engaging end of the passenger boarding bridge relative a horizontal reference plane.
17. A method according to claim 14, wherein sensing orientation data comprises sensing a distance between the aircraft-engaging end of the passenger boarding bridge and the aircraft.
18. A method according to claim 14, wherein the template image data comprises data relating to at least an image including a feature that is indicative of the location of the doorway of the aircraft.
19. A method according to claim 18, wherein comparing the captured image data with the transformed template image data comprises determining a correlation between the feature that is indicative of the location of the doorway of the aircraft in the transformed template image data and a corresponding feature in the captured image data.
20. A method according to claim 14, comprising guiding the aircraft to the defined parking space using a visual docking guidance system (VDGS).
21. A method according to claim 20, comprising using a sensor of the VDGS for identifying the type and subtype of the aircraft.
22. A method for aligning an aircraft-engaging end of a passenger boarding bridge with a doorway of an aircraft, comprising:
- using a visual docking guidance system (VDGS), guiding the aircraft to a predetermined stopping position adjacent to the passenger boarding bridge;
- positioning the aircraft-engaging end of the passenger boarding bridge at a known position for capturing an image of a portion of the aircraft that is proximate an expected stopping location of the doorway;
- using an imager disposed proximate the aircraft-engaging end of the passenger boarding bridge, capturing an image of the portion of the aircraft that is proximate the expected stopping location of the doorway;
- sensing orientation data relating to tilt of the aircraft-engaging end of the passenger boarding bridge relative a horizontal reference plane, and relating to distance between the aircraft-engaging end of the passenger boarding bridge and the portion of the aircraft that is proximate the expected stopping location of the doorway;
- retrieving a template image from a memory element, the template image relating to the doorway of the aircraft;
- transforming the template image based on the sensed orientation data;
- based on a comparison between the transformed template image and the captured image, determining instruction data for moving the aircraft-engaging end of the passenger boarding bridge into an aligned condition with the doorway of the aircraft from the known position; and, under the control of an automated control system of the passenger boarding bridge, moving the aircraft-engaging end of the passenger boarding bridge in accordance with the instruction data, so as to align the aircraft-engaging end of the passenger boarding bridge with the doorway of the aircraft.
23. A method according to claim 22, wherein moving the aircraft-engaging end of the passenger boarding bridge in accordance with the instruction data is performed without capturing an additional image during the course of the movement.
24. A method according to claim 22, comprising determining a type and sub-type of the aircraft, wherein the known position is a predetermined photo position for the determined type and sub-type of the aircraft.
25. A method according to claim 24, wherein the instruction data includes at least horizontal movement instruction data and vertical movement instruction data for effecting a displacement of the aircraft-engaging end of the passenger boarding bridge from the predetermined photo position.
26. A method according to claim 25, comprising correcting the vertical movement instruction based on a bridge specific correction factor.
27. A method according to claim 26, wherein the bridge specific correction factor is based on a known ground surface elevation profile between the photo position and the predetermined stopping position of the aircraft.
28. A method for aligning an aircraft-engaging end of a passenger boarding bridge with a doorway of an aircraft, the aircraft stopped within a defined parking space adjacent to the passenger boarding bridge, the method comprising:
- moving the aircraft-engaging end of the passenger boarding bridge to a known position adjacent to an expected stopping location of the doorway of the aircraft;
- capturing first image data using a first imager that is disposed proximate the aircraft-engaging end of the passenger boarding bridge, the first image data relating to a first portion of the aircraft that is proximate the expected stopping location of the doorway;
- capturing second image data using a second imager disposed proximate the aircraft-engaging end of the passenger boarding bridge, the second image data relating to a second portion of the aircraft that is proximate the expected stopping location of the doorway;
- sensing orientation data including tilt data, the orientation data relating to an orientation of the aircraft-engaging end of the passenger boarding bridge after moving to the known position;
- retrieving template image data relating to the doorway of the aircraft;
- transforming the template image. data based on the sensed orientation data;
- comparing separately the captured first image data and the captured second image data with the transformed template image data, so as to determine independently first alignment data and second alignment data, respectively;
- selecting one of the first alignment data and the second alignment data based on a predefined selection criterion; and
- aligning the aircraft-engaging end of the passenger boarding bridge with the doorway of the aircraft based upon the selected one of the first alignment data and the second alignment data.
29. A method according to claim 28, comprising determining a type and sub-type of the aircraft, wherein the known position is a predetermined photo position for the determined type and sub-type of the aircraft.
30. A method according to claim 29, wherein the selected one of the first alignment data and the second alignment data includes at least horizontal movement instruction data and vertical movement instruction data for effecting a displacement of the aircraft-engaging end of the passenger boarding bridge from the predetermined photo position.
31. A method according to claim 30, comprising correcting the vertical movement instruction based on a bridge specific correction factor.
32. A method according to claim 31, wherein the bridge specific correction factor is based on a known ground surface elevation profile between the photo position and the predetermined stopping position of the aircraft.
33. A method according to claim 28, wherein sensing orientation data including tilt data comprises using an inclinometer disposed proximate the aircraft-engaging end of the passenger boarding bridge for sensing tilt of the aircraft-engaging end of the passenger boarding bridge relative a horizontal reference plane.
34. A method according to claim 28, wherein sensing orientation data comprises sensing a distance between the aircraft-engaging end of the passenger boarding bridge and the aircraft.
Type: Application
Filed: Oct 31, 2006
Publication Date: May 1, 2008
Applicant: DEW Engineering and Development Limited (Ottawa)
Inventor: Neil Hutton (Ottawa)
Application Number: 11/589,737
International Classification: E01D 1/00 (20060101);