Use of a Single Camera for Multiple Driver Assistance Services, Park Aid, Hitch Aid and Liftgate Protection
The present invention is a system for providing multiple driver assistance services which includes a vehicle having at least one door, and at least one imaging device operable for detecting the presence of one or more objects in proximity to the door for providing all distances between a vehicle and one or more objects in proximity to the vehicle. The imaging device is operable for displaying an image representing the various objects.
The instant application claims priority to U.S. Provisional Patent Application Ser. No. 61/011,795, filed Jan. 22, 2008, the entire specification of which is expressly incorporated herein by reference.
FIELD OF THE INVENTIONThe present invention relates to an object detection system, and to a method using an algorithm to process three dimensional data imaging for object tracking and ranging; more particularly, the present invention uses a single camera for providing multiple driver assistance services, such as park aid, hitch aid, and liftgate protection.
BACKGROUND OF THE INVENTIONVehicle park-aid systems are generally known and are commonly used for the purpose of assisting vehicle operators in parking a vehicle by alerting the operator of potential parking hazards. Typical park-aid systems include ultrasonic or camera systems. Ultrasonic systems can alert the vehicle operator of the distance between the vehicle and the closest particular object. However, ultrasonic systems do not recognize what the objects are and also fail to track multiple objects at the same time. Camera systems can present the vehicle operator with the view from behind the vehicle, however, camera systems do not provide the operator with the distance to the objects viewed and do not differentiate whether or not the viewed objects are within the vehicle operator's field of interest.
Also, the use of multiple three-dimensional imagers for multiple applications is not cost effective. The operations of providing park-aid, hitch-aid, and liftgate protection have been attempted individually, but not by a single system. Also, camera-based environment sensing is unable to alert the driver of objects of interest within the field of view of the camera or three-dimensional imager. The driver must watch the screen and decide which objects present a risk to the vehicle. Non-camera based systems do not provide a view of the environment and don't allow the same visibility provided by a camera system.
Accordingly, there exists a need for a more advanced object detection and ranging system which can filter and process data provided by a three dimensional camera to provide an effective translation of object information to a vehicle operator that can be used in providing assistance to a driver when performing certain tasks, such as parking (i.e. a park aid), attaching a trailer to the hitch of a vehicle (i.e. a hitch aid), or opening and closing a liftgate (i.e. liftgate protection).
SUMMARY OF THE INVENTIONThe present invention is directed to a method of object detection and ranging of objects within a vehicle's field of interest and providing a translation of the object data to a vehicle operator, as well as providing park aid, a hitch aid, and liftgate protection. This is accomplished by providing a camera-based interface that will alert the driver of objects of interest within the field of view while still providing the full view of the environment. An imaging device provides an image of the rearward area outside of a vehicle to a data processor. The processor divides the data into individual rows of pixels for processing, and uses an algorithm which includes assigning each pixel in the rows to an object that was detected by the imaging device; this allows for a real world translation of detected objects and their respective coordinates, including dimensions and distance from the vehicle. The location of the detected objects is available to the vehicle operator to provide a detailed warning of objects within the field of interest.
By aiming the imaging device(s) properly to view the field behind the vehicle, it is possible to perform all the functions mentioned above by a single imaging device system. The operation of the system is determined based on vehicle gear state, liftgate position, liftgate movement, vehicle speed and user input. The functions that the system can perform include, but are not limited to: sensing the environment behind the vehicle and warning the driver through audible or visual feedback of objects; detecting objects in the path of the moving liftgate during opening and closing; warning the driver of potential collisions through audio or visual feedback; and stopping the movement of the liftgate; recognizing a trailer and tracking the position of the trailer relative to vehicle's trailer hitch to aid the driver in the process of maneuvering the vehicle to hooking up the trailer by audible feedback, visual feedback, or a combination of both when backing-up.
The present invention is a system for providing multiple driver assistance services which includes a vehicle having at least one door, and at least one imaging device operable for detecting the presence of one or more objects in proximity to the door for providing all distances between a vehicle and one or more objects in proximity to the vehicle. The imaging device is operable for displaying an image representing the one or more objects.
Further areas of applicability of the present invention will become apparent from the detailed description provided hereinafter. It should be understood that the detailed description and specific examples, while indicating the preferred embodiment of the invention, are intended for purposes of illustration only and are not intended to limit the scope of the invention.
The present invention will become more fully understood from the detailed description and the accompanying drawings, wherein:
The following description of the preferred embodiment(s) is merely exemplary in nature and is in no way intended to limit the invention, its application, or uses.
Referring to
Referring to
The row processing algorithm shown at 16 generally has four processing steps each including the use of a respective equation, wherein completion of the four processing steps allows the current pixel being evaluated, herein called a “pixel of interest,” to be assigned to an object. A first processing step 26 and a second processing step 28 are threshold comparisons based on different criteria and equations. The first processing step 26 and second processing step 28 can use equation 1 and equation 2 respectfully. A third processing step 30 and a fourth processing steps 32 are spatial operations based on different criteria and equations performed on the pixel of interest. The third processing step 30 and fourth processing step 32 can use equation 3 and equation 4 respectfully. The first and second processing steps 26,28 must be performed before the third and fourth processing steps 30,32 as data from the first and second processing steps 26,28 is required for the third and fourth processing steps 30,32. Outlined below are samples of equations 1 and 2 used in carrying out the first and second processing steps 26,28 respectively and equations 3 and 4 used in carrying out the third and fourth processing steps 30,32 respectively.
The first and second processing steps 26,28 are generally filtering or point based operations which operate on a pixel disposed one row ahead and one column ahead of the pixel of interest being evaluated for assignment to an object. The first processing step 26 uses equation 1 and includes comparing a confidence map to a minimum confidence threshold. The first processing step 26 determines a confidence factor for each pixel of the collected frame to show reliability of the pixel data collected along the z-axis. The confidence factor is compared to a static threshold, e.g., a predetermined constant, and the data is filtered. The second processing step 28 uses equation 2 and includes comparing distance data to ground threshold data. The second processing step 28 compares the data, e.g., pixel data, collected along the z-axis to a pixel map of a surface, e.g., the ground surface rearward of the vehicle upon which the vehicle travels. This allows the surface, e.g., ground surface, in the captured image to be filtered out or ignored by the algorithm. It is understood that additional surfaces or objects, e.g., static objects, the vehicle bumper, hitch, rear trim, and the like, can be included in the pixel map of the surface such that they too can be filtered out or discarded by the algorithm.
The third and fourth processing steps 30,32 are generally spatial operations or processes performed on the pixel of interest in order to assign the pixel of interest to an object. The third processing step 30 uses equation 3 and is a morphological erosion filter used to eliminate and discard single pixel noise, e.g., an invalid, inaccurate, unreliable, and the like pixel of interest. This step requires that the data in the forward adjacent pixels, e.g., r+m, c+n, of the collected frame be present and valid in order for the pixel of interest to be valid. The fourth processing step 32 uses equation 4 and includes a three dimensional (“3D”) connected components algorithm which groups together objects based on a minimum distance between the z-axis data of the pixel of interest and the z-axis data of pixels adjacent to the pixel of interest which have already been assigned to objects. The 3D connected components algorithm requires that the pixel of interest be compared to the backward pixels, e.g., r-m, c-n. Equation 4 can depict the result of the algorithm, however, it is understood that the implementation can differ. By way of non-limiting example, equation 4 can ignore the merging of objects, e.g., of step 20, and assign pixels of interest to new objects and re-assign the pixels if necessary.
Referring generally to
Referring generally to
At first decision 122 of the pixel connected components algorithm 40, if and only if the object to which a pixel of comparison was assigned is not valid, e.g., deemed invalid by third processing step 30, not yet assigned, is pixel noise, and the like, then a new object flag is set for the object to which the pixel of interest, (“r,c”), was assigned at State 3. If the object to which a pixel of comparison was assigned is valid, then second decision 124 is performed. At second decision 124, if the depth data for the pixel of interest, “Z(r,c)”, minus the depth data for the pixel of comparison, “Z(POC)” is less than the minimum distance, then third decision 126 is performed, e.g., minimum distance between the z-axis data of the pixel of interest and the z-axis data of pixels adjacent to the pixel of interest. If not, then the object to which the pixel of interest was assigned is set or flagged as new at state 1. At third decision 126, if and only if the object to which the pixel of interest was assigned is valid, then the processor either selectively assigns the object to which the pixel of interest was assigned to the object to which the object to with the pixel of comparison was assigned at state 1, or selectively merges the object to which the pixel of interest was assigned with the object to which the pixel of comparison was assigned at state 2 (shown in
Referring to
Referring to
Referring to
The ability to depict various objects in proximity to the vehicle has many types of applications, such as aiding the driver of the vehicle in parking (park aid), aiding in the attachment of a hitch to the rear of the vehicle (hitch aid), and protecting the liftgate from contacting objects when opening (liftgate protection).
Referring to
The camera 56 provides all of the information to the driver on a display as a monochrome image 72, shown in
The image 72 from the camera 56 is collected by a suitable digital signal processor (DSP) and is processed by an object detection algorithm (as described above) or some filtering process to find objects of interest to the driver. The raw data is then converted to a monochrome image (if necessary). The objects found by the DSP are then highlighted according to distance in the given image using the pixels similar to the pixels 74,76 shown in
The system provides several advantages. The system is used for interpolation of distance into varying colors of the pixels 72,74 in a fashion that provides for variable driver warning within a distance measuring and imaging system. The system can be integrated into the rear end of the vehicle 60. The camera 56 is not limited to being integrated with the deck lid 58, as described above, but could also be integrated with the light gate, spoiler, or fascia. The system senses objects entering the area of interest behind the vehicle 60 and warns the driver through audible, visual or both indicators when backing-up. Additionally, the system senses objects on the path of the power lift gate 66 as the liftgate 66 swings up or down and prevents the liftgate 66 from touching the objects on its path. Also, the system recognizes a trailer 68 and tracks the position of the vehicle 60 relative to the trailer hitch 70 and aids the driver in the process of maneuvering the vehicle while hooking up the trailer 68 by audible, visual or both indicators when backing-up.
The description of the invention is merely exemplary in nature and, thus, variations that do not depart from the gist of the invention are intended to be within the scope of the invention. Such variations are not to be regarded as a departure from the spirit and scope of the invention.
Claims
1. A system for providing multiple driver assistance services, comprising:
- a vehicle having at least one door; and
- at least one imaging device operable for detecting the presence of one or more objects in proximity to said door for providing all distances between a vehicle and one or more objects in proximity to said vehicle, said at least one imaging device being operable for displaying an image representing said one or more objects.
2. The system for providing multiple driver assistance services of claim 1, further comprising at least one detection zone, wherein said at least one imaging device is operable to detect said one or more objects in said at least one detection zone.
3. The system for providing multiple driver assistance services of claim 2, said image representing said one or more objects in said at least one detection zone, indicative of the distance between said one or more objects and said at least one door.
4. The system for providing multiple driver assistance services of claim 1, further comprising a digital signal processor operable with an object detection algorithm for collecting said image produced by said imaging device.
5. The system for providing multiple driver assistance services of claim 4, wherein said digital signal processor and said object detection algorithm are operable for dividing said image into a plurality of pixels.
6. The system for providing multiple driver assistance services of claim 5, further comprising each of said plurality of pixels being designated a specific color, said plurality of pixels being divided into one or more groups of pixels, each of said one or more groups of pixels having substantially similar colors.
7. The system for providing multiple driver assistance services of claim 6, said each of said one or more groups of pixels having a substantially similar color being operable for providing an indication of the location of said one or more objects in relation to said at least one door of said vehicle.
8. The system for providing multiple driver assistance services of claim 4, wherein one of said multiple driver assistance services is a park aid, comprising:
- said at least one door further comprising a liftgate; and
- wherein as said one or more objects enters said at least one detection zone during the movement of said liftgate, said digital signal processor and said object detection algorithm are operable to perform one selected from the group consisting of reporting the position of said one or more objects to a user of said vehicle, halting the movement of said liftgate, reversing the movement of said liftgate, and combinations thereof.
9. The system for providing multiple driver assistance services of claim 4, wherein one of said multiple driver assistance services is aiding in the attachment of a trailer to said vehicle, comprising:
- a trailer; and
- a hitch connected to said trailer, said hitch being operable for connection with said vehicle, wherein as said hitch moves in said at least one detection zone as said vehicle moves towards said trailer, said digital signal processor and said object detection algorithm are operable for calculating the trajectory required for said vehicle to properly align with said hitch.
10. The system for providing multiple driver assistance services of claim 1, wherein said at least one imaging device is connected to component of said vehicle, said component selected from the group consisting of a light gate, a deck lid, a spoiler, a fascia, and combinations thereof.
11. A system for providing multiple driver assistance services, comprising:
- a vehicle having at least one door which is moveable between an open position and a closed position;
- at least one imaging device used for detecting the presence of various objects in proximity to said door; and
- a detection zone, said at least one imaging device being operable for detecting one or more objects in said detection zone, for providing an indication of the position of said one or more objects in relations to said vehicle.
12. The system for providing multiple driver assistance services of claim 11, further comprising an image operable for being displayed by said at least one imaging device, said image representing said one or more objects in said detection zone.
13. The system for providing multiple driver assistance services of claim 12, further comprising a digital signal processor operable with an object detection algorithm for collecting said image produced by said imaging device, wherein said digital signal processor and said object detection algorithm are operable for dividing said image into a plurality of pixels.
14. The system for providing multiple driver assistance services of claim 13, wherein each of said plurality of pixels is designated to be a specific color, wherein said plurality of pixels are divided into one or more groups of pixels.
15. The system for providing multiple driver assistance services of claim 14, wherein each of said one or more groups of pixels are of substantially similar colors.
16. The system for providing multiple driver assistance services of claim 15, wherein each of said one or more groups of pixels having a substantially similar color provide an indication of the location of said one or more objects in said detection zone in proximity to said vehicle.
17. The system for providing multiple driver assistance services of claim 13, wherein one of said multiple driver assistance services is a park aid, comprising:
- said at least one door further comprising a liftgate; and
- wherein as said one or more objects enters said detection zone during the movement of said liftgate, said digital signal processor and said object detection algorithm are operable to perform one selected from the group consisting of reporting the position of said one or more objects to a user of said vehicle, halting the movement of said liftgate, reversing the movement of said liftgate, and combinations thereof.
18. The system for providing multiple driver assistance services of claim 13, wherein one of said multiple driver assistance services is aiding in the attachment of a trailer to said vehicle, comprising:
- a trailer; and
- a hitch connected to said trailer, said hitch being operable for connection with said vehicle, wherein as said hitch moves in said detection zone as said vehicle moves towards said trailer, said digital signal processor and said object detection algorithm are operable for calculating the trajectory required for said vehicle to properly align with said hitch.
19. The system for providing multiple driver assistance services of claim 11, wherein said at least one imaging device is connected to a component of said vehicle, said component selected from the group consisting of a light gate, a deck lid, a spoiler, a fascia, and combinations thereof.
20. A system for providing multiple driver assistance services, comprising:
- a vehicle having at least one door which is moveable between an open position and a closed position, and any position therebetween;
- at least one imaging device used for detecting the presence of various objects in proximity to said door;
- a detection zone, said at least one imaging device being operable for detecting one or more objects in said detection zone; and
- an image displayed by said at least one imaging device, said image representing said one or more objects in said detection zone, thereby indicating to a user of said vehicle where said one or more objects are in proximity to said at least one door.
21. The system for providing multiple driver assistance services of claim 20, further comprising a digital signal processor operable with an object detection algorithm for collecting said image produced by said imaging device, and dividing said image into a plurality of pixels.
22. The system for providing multiple driver assistance services of claim 21, wherein one of said multiple driver assistance services is a park aid, comprising:
- said at least one door further comprising a liftgate; and
- wherein as said one or more objects enters said detection zone during the movement of said liftgate, said digital signal processor and said object detection algorithm are operable to perform one selected from the group consisting of reporting the position of said one or more objects to a user of said vehicle, halting the movement of said liftgate, reversing the movement of said liftgate, and combinations thereof.
23. The system for providing multiple driver assistance services of claim 21, wherein one of said multiple driver assistance services is aiding in the attachment of a trailer to said vehicle, comprising:
- a trailer; and
- a hitch connected to said trailer, said hitch being operable for connection with said vehicle, wherein as said hitch moves in said detection zone as said vehicle moves towards said trailer, said digital signal processor and said object detection algorithm are operable for calculating the trajectory required for said vehicle to properly align with said hitch.
24. The system for providing multiple driver assistance services of claim 21, wherein each of said plurality of pixels is of a specific color, and said plurality of pixels are divided into one or more groups of pixels, each of said one or more groups of pixels having substantially similar colors.
25. The system for providing multiple driver assistance services of claim 24, wherein each of said one or more groups of pixels having a similar color provide an indication of the location of each of said one or more objects in said detection zone in proximity to said vehicle.
26. The system for providing multiple driver assistance services of claim 20, wherein said at least one imaging device is connected to a component of said vehicle, said component selected from the group consisting of a light gate, a deck lid, a spoiler, a fascia, and combinations thereof.
Type: Application
Filed: Jan 21, 2009
Publication Date: Feb 24, 2011
Inventors: Guner R. Sarioglu (Oakland Township, MI), John T. Hargenrader (Brighton, MI), Matthew T. Burtch (Concord, CA)
Application Number: 12/812,828
International Classification: H04N 7/18 (20060101);