LANE RECOGNITION DEVICE AND METHOD THEREOF
A lane recognition device includes: a camera configured to capture an image in front of a vehicle; and a controller configured to detect a lane from the image in front of the vehicle; generate a plurality of lane equations based on a curved point of the lane; and recognize the lane based on the plurality of lane equations.
This application claims the benefit of priority to Korean Patent Application No. 10-2019-0007656, filed in the Korean Intellectual Property Office on Jan. 21, 2019, the entire contents of which are incorporated herein by reference.
TECHNICAL FIELDThe present disclosure relates to technologies of recognizing lanes.
BACKGROUNDAs available functions in a vehicle become more advanced, vehicles with various safety systems have been increasing. Various safety systems for vehicles have been developed to detect accidents occurring when driving or parking, by using various sensors, a vision system, and a laser system, so as to warn a driver of the accidents or to control the vehicle. Such safety systems include, for example, an electric stability program (ESP), an adaptive cruise control (ACC), a lane keeping assist system (LKAS), a lane departure warning system (LDWS), and the like.
Currently available safety systems can recognize lanes and provide services, such as inter-vehicle distance keeping and lane keeping, based on the recognized lanes.
For instance, a conventional lane recognition technology is developed for capturing an image in front of a vehicle by means of a camera loaded into the vehicle, detecting a lane from the captured image in front of the vehicle, performing curve fitting of a set of location coordinates (x, y) of the detected lane to approximate the set of the location coordinates (x, y) to a cubic equation of the lane, and recognizing the lane based on the cubic equation. The cubic equation of the approximated lane is Equation 1 below.
y=c3x3+c2x2+c1x+c0 [Equation 1]
Here, C0 denotes the lateral offset of the lane, C1 denotes the heading angle slope of the lane, C2 denotes the curvature of the lane, and C3 denotes the curvature rate of the lane. The lateral offset of the lane is a parameter indicating how far the lane is located from the center of the vehicle, and the heading angle slope of the lane is a parameter indicating a slope between the lane and a progress direction of the vehicle.
Such a conventional lane recognition technology represents a real lane as a cubic equation of the lane without considering a case where the lane on the road is curved to form a certain angle without being bent in the form of a smooth curve (e.g., a lane in a form where two straight lines from one point form the certain angle). Such that an error between the real lane and the cubic equation of the lane (the recognized lane) is increased, and thus, the driving safety is not ensured by the conventional technology.
SUMMARYThe present disclosure has been made to solve the above-mentioned problems occurring in the prior art while advantages achieved by the prior art are maintained intact.
An aspect of the present disclosure provides a lane recognition device for recognizing a lane with high accuracy when a lane on the road is curved to form a certain angle without being bent in the form of a smooth curve (e.g., in case of a lane in a form where two straight lines from one point form the certain angle), by representing the lane as a plurality of cubic equations in consideration of a form of the lane drawn on the road and a method thereof.
The technical problems to be solved by the present disclosure are not limited to the aforementioned problems, and any other technical problems not mentioned herein will be clearly understood from the following description by those skilled in the art to which the present disclosure pertains.
According to an aspect of the present disclosure, a lane recognition device may include: a camera configured to capture an image in front of a vehicle; and a controller configured to recognize a lane based on a curved point of the lane obtained from the image in front of the vehicle.
The controller may be configured to detect the lane from the image in front of the vehicle, the image being captured by the camera, generate a plurality of lane equations on the basis of the curved point of the lane, and recognize the lane on a road based on the plurality of lane equations.
The controller may be configured to convert the detected lane from a camera coordinate system to a road coordinate system.
The controller may be configured to detect the lane in a setting interval of a minimum point to a maximum point in the image in front of the vehicle, approximate the detected lane to a lane equation, and detect the curved point, when an error between the lane and the lane equation is greater than a first threshold. The error may be an area formed by the lane and the lane equation.
The controller may be configured to generate a first error graph between the lane and the lane equation in a process of performing curve fitting of a lane set configuring the detected lane in ascending order of x values, generate a second error graph between the lane and the lane equation in a process of performing curve fitting of the lane set in descending order of the x values, and set a point where the first error graph and the second error graph are met as the curved point.
The controller may be configured to generate a first lane equation by performing curve fitting of a lane set in an interval of the minimum point to the curved point in the ascending order of the x values and generate a second lane equation by performing curve fitting of a lane set in an interval of the maximum point to the curved point in the descending order of the x values.
The lane recognition device may further include a radar configured to detect a location and a relative speed of a preceding vehicle. The controller may be configured to calculate a driving trajectory based on the location and the relative speed of the preceding vehicle, perform curve fitting of a set configuring the calculated driving trajectory, and reduce the first threshold, when a curvature rate of the driving trajectory is greater than a second threshold.
The lane recognition device may further include a communicator configured to obtain a driving trajectory of a road where the vehicle is currently traveling from a driving trajectory server. The controller may be configured to perform curve fitting of a set configuring the driving trajectory obtained via the communicator and reduce the first threshold, when a curvature rate of the driving trajectory is greater than a second threshold.
According to another aspect of the present disclosure, a lane recognition method may include: capturing, by a camera of a vehicle, an image in front of the vehicle; and recognizing, by a controller of the vehicle, a lane based on a curved point of the lane obtained from the image in front of the vehicle.
The recognizing of the lane may include detecting the lane from the image in front of the vehicle, generating a plurality of lane equations on the basis of the curved point of the lane, and recognizing the lane on a road based on the plurality of lane equations. The detecting of the lane may include converting the detected lane from a camera coordinate system to a road coordinate system.
The calculating of the plurality of equations may include detecting the lane in a setting interval of a minimum point to a maximum point in the image in front of the vehicle, approximating the detected lane to a lane equation, and detecting the curved point, when an error between the lane and the lane equation is greater than a first threshold. It is preferable that the error is an area formed by the lane and the lane equation.
The detecting of the curved point may include generating a first error graph between the lane and the lane equation in a process of performing curve fitting of a lane set configuring the detected lane in ascending order of x values, generating a second error graph between the lane and the lane equation in a process of performing curve fitting of the lane set in descending order of the x values, and setting a point where the first error graph and the second error graph are met as the curved point.
The calculating of the plurality of lane equations may include generating a first lane equation by performing curve fitting of a lane set in an interval of the minimum point to the curved point in the ascending order of the x values and generating a second lane equation by performing curve fitting of a lane set in an interval of the maximum point to the curved point in the descending order of the x values.
The calculating of the plurality of equations may include calculating a driving trajectory based on a location and a relative speed of a preceding vehicle, performing curve fitting of a set configuring the calculated driving trajectory, and reducing the first threshold, when a curvature rate of the driving trajectory is greater than a second threshold.
The calculating of the plurality of equations may include obtaining a driving trajectory of a road where the vehicle is currently traveling from a driving trajectory server, performing curve fitting of a set configuring the driving trajectory, and reducing the first threshold, when a curvature rate of the driving trajectory is greater than a second threshold.
The above and other objects, features and advantages of the present disclosure will be more apparent from the following detailed description taken in conjunction with the accompanying drawings:
Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the exemplary drawings. In adding the reference numerals to the components of each drawing, it should be noted that the identical or equivalent component is designated by the identical numeral even when they are displayed on other drawings. Further, in describing the embodiment of the present disclosure, a detailed description of well-known features or functions will be ruled out in order not to unnecessarily obscure the gist of the present disclosure.
In describing the components of the embodiment according to the present disclosure, terms such as first, second, “A”, “B”, (a), (b), and the like may be used. These terms are merely intended to distinguish one component from another component, and the terms do not limit the nature, sequence or order of the constituent components. Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meanings as those generally understood by those skilled in the art to which the present disclosure pertains. Such terms as those defined in a generally used dictionary are to be interpreted as having meanings equal to the contextual meanings in the relevant field of art, and are not to be interpreted as having ideal or excessively formal meanings unless clearly defined as having such in the present application.
The lane 30 may not be bent in a smooth curve but is curved at a certain angle at point 50. In other words, the lane 30 may have a form where two straight lines form a certain angle on the basis of the division point 50. For reference, there is a limitation that a cubic equation accurately expresses such a lane 30.
In
Furthermore, a coordinate system (x, y) displayed on the host vehicle 10 is a coordinate system of the camera loaded into the host vehicle 10 and is a criterion of indicating locations of objects in an image in front of the host vehicle 10, captured by the camera. Herein, x denotes a longitudinal direction of the host vehicle 10, and y denotes a lateral direction of the host vehicle 10.
As shown in
Seeing the respective components, the memory 210 may store various logics, algorithms, and programs necessary to represent a lane as a plurality of cubic equations in consideration of a form of the lane drawn on a road.
Furthermore, the memory 210 may store a curve fitting technique which approximates a real lane to a cubic equation based on a least square method or the like.
Such a memory 210 may include at least one type of storage medium, such as a flash memory type memory, a hard disk type memory, a micro type memory, a card type memory (e.g., a secure digital (SD) card or an extreme digital (XD) card), a random access memory (RAM), a static RAM (SRAM), a read-only memory (ROM), a programmable ROM (PROM), an electrically erasable PROM (EEPROM), a magnetic RAM (MRAM), a magnetic disk, and an optical disk.
The camera 220 may be a module, loaded into a vehicle, for capturing an image in front of the vehicle, and may have an image capture range corresponding to reference numeral 11 shown in
The radar 230 may be a module, loaded into the vehicle, for detecting an area in front of the vehicle, and may have a detection area corresponding to reference numeral 12 shown in
In addition, the radar 230 may play a role in assisting the camera 220. In other words, data obtained by the radar 230 may be used additionally when detecting a lane from the image in front of the vehicle, captured by the camera 220.
The GPS receiver 240 may be a module for receiving location information (coordinates) from a GPS satellite and may be used to identify a current location of a host vehicle 10 of
The communicator 250 may be a module for providing a communication interface with a driving trajectory server 300 and may obtain a driving trajectory for a road where the host vehicle 10 is traveling from the driving trajectory server 300. In this case, the driving trajectory may be a driving trajectory of a vehicle which previously traveled on the road, or may be an optimal driving trajectory derived from driving trajectories (a plurality of driving trajectories) of vehicles which previously traveled on the road. For example, the optimal driving trajectory may be derived from a movement average of the vehicles which previously traveled on the road.
Such a communicator 250 may have a mobile communication module, a wireless Internet module, and a short-range communication module.
Herein, the mobile communication module may transmit and receive a wireless signal with the driving trajectory server 300 over a mobile communication network which is established according to a technical standard or a communication mode for mobile communication (e.g., global system for mobile communication (GSM), code division multiple access (CDMA), code division multiple access 200 (CDMA2000), enhanced voice-data optimized or enhanced voice-data only (EV-DO), wideband CDMA (WCDMA), high speed downlink packet access (HSDPA), high speed uplink packet access (HSUPA), long term evolution (LTE), LTE-advanced (LTE-A), or the like). The wireless signal may include a voice call signal, a video call signal, or data in various forms according to transmission and reception of a text/multimedia message.
The wireless Internet module may be configured to transmit and receive a wireless signal on a communication network according to wireless Internet technology. The wireless Internet technology may be, for example, wireless local area network (WLAN), wireless-fidelity (Wi-Fi), Wi-Fi Direct, digital living network alliance (DLNA), wireless broadband (WiBro), world interoperability for microwave access (WiMAX), high speed downlink packet access (HSDPA), high speed uplink packet access (HSUPA), long term evolution (LTE), LTE-advanced (LTE-A), or the like.
The short-range communication module may be for short-range communication and may support the short-range communication using at least one of Bluetooth™, radio frequency identification (RFID), infrared data association (IrDA), ultra wideband (UWB), ZigBee, near field communication (NFC), Wi-Fi, Wi-Fi Direct, and wireless universal serial bus (USB) technologies.
The controller 260 may perform overall control such that the respective components normally perform their own functions. Such a controller 260 may be implemented in the form of hardware or software or in the form of a combination thereof. The controller 260 may be implemented as, but not limited to, a microprocessor, or may be an electronic control unit (ECU).
Such a controller 260 may approximate a lane to a plurality of cubic equations in consideration of the lane drawn on a road and may perform a variety of control necessary to perform the process of recognizing the lane on the road based on the approximated cubic equations.
Hereinafter, a description will be given in detail of an operation of the controller 260 with reference to
The controller 260 may detect a lane from an image in front of a vehicle, captured by the camera 220. The detected lane may be represented as a set of a camera coordinate system (i, j).
Thereafter, the controller 260 may convert the lane set represented as the camera coordinate system into a road coordinate system (x, y).
As shown in
y(i,j)=X(j)×tan−1(Ap×(i−Ch)) [Equation 2]
Here, x denotes the x coordinate value of the road, y denotes the y coordinate value of the road, Ch denotes the horizontal vanishing point of the camera, i denotes the horizontal pixel of the camera, j denotes the vertical pixel of the camera, and Ap denotes the angle of view per one pixel. In this case, Ap satisfies Equation 3 below, and X(j) satisfies Equation 4 below.
Here, Fh denotes the horizontal angle of view, Fv denotes the vertical angle of view, Ph denotes the horizontal resolution, and Pv denotes the vertical resolution.
Here, Hc denotes the height of the camera, and Cv denote the vertical vanishing point the camera.
The controller 260 may convert an image in a camera coordinate system into an image in a road coordinate system based on Equation 2 above. In this case, because a lane is included in the image in the camera coordinate system, the lane in the camera coordinate system may be converted into a lane in the road coordinate system in the conversion process. The controller 260 may detect the lane from the image in the camera coordinate system and may convert only the detected lane into a lane in the road coordinate system, without converting the entire image into an image in the road coordinate system.
Thereafter, the controller 260 may perform a process of determining whether a curved point 50 of
To determine whether to detect a curved point on a lane, as shown in
When approximating the real lanes 310 and 320 to the cubic equation 330 of the lane for the setting interval, when the error Ea between the cubic equation 330 of the lane and the real lanes 310 and 330 is greater than a threshold Eth, the controller 260 may determine that there is the curved point 50 to perform a process of detecting the curved point 50. In this case, a size of the error may be represented as a size of an area formed by the real lanes 310 and 320 and the cubic equation 330 of the lane.
Thereafter, as shown in
Furthermore, as shown in
Thereafter, as shown in
The controller 260 may detect a point dc at which a first error graph and a second error graph are met as the curved point 50.
Thereafter, the controller 260, as shown in
Comparing
The controller 260 may detect a driving trajectory 21 of a preceding vehicle 20 of
Thereafter, the controller 260 may perform approximation (curve fitting) of the driving trajectory 21 of the preceding vehicle 20 to the cubic equation. The approximated cubic equation may be Equation 5 below.
y=p3x3+p2x2+p1x+p0 [Equation 5]
Here, P0 denotes the lateral offset of the driving trajectory, P1 denotes the heading angle slope of the driving trajectory, P2 denotes the curvature of the driving trajectory, and P3 denotes the curvature rate of the trajectory.
Thereafter, when P3 is greater than a threshold Eth, the controller 260 may multiply the threshold Eth by a weight w to set the threshold Eth to a smaller value. In this case, it is preferable that the weight w is a value between 0 and 1.
Meanwhile, the controller 260 may obtain a driving trajectory of a road where the vehicle is currently traveling, obtained from a driving trajectory server 300 of
In operation 701, a camera 220 of
In operation 702, a controller 260 of
In operation 703, the controller 260 may generate a plurality of lane equations on the basis of the curved point of the lane. In other words, the controller 260 may perform curve fitting of the lane set (x, y) in an interval of dmin to dc in the ascending order of the x values to generate a first lane equation and may perform curve fitting of the lane set (x, y) in an interval of dmax to dc in the descending order of the x values to generate a second lane equation.
In operation 704, the controller 260 may recognize the lane on a road based on the plurality of lane equations.
Referring to
The processor 1100 may be a central processing unit (CPU) or a semiconductor device that processes instructions stored in the memory 1300 and/or the storage 1600. The memory 1300 and the storage 1600 may include various types of volatile or non-volatile storage media. For example, the memory 1300 may include a ROM (Read Only Memory) and a RAM (Random Access Memory).
Thus, the operations of the method or the algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware or a software module executed by the processor 1100, or in a combination thereof. The software module may reside on a storage medium (that is, the memory 1300 and/or the storage 1600) such as a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM. The exemplary storage medium may be coupled to the processor 1100, and the processor 1100 may read information out of the storage medium and may record information in the storage medium. Alternatively, the storage medium may be integrated with the processor 1100. The processor 1100 and the storage medium may reside in an application specific integrated circuit (ASIC). The ASIC may reside within a user terminal. In another case, the processor 1100 and the storage medium may reside in the user terminal as separate components.
The lane recognition device and the method thereof according to an exemplary embodiment of the present disclosure may recognize the lane with high accuracy when the lane on the road is curved to form a certain angle without being bent in the form of a smooth curve (e.g., in case of a lane in a form where two straight lines from one point form the certain angle), by representing the lane as a plurality of cubic equations in consideration of a form of the lane drawn on the road.
Hereinabove, although the present disclosure has been described with reference to exemplary embodiments and the accompanying drawings, the present disclosure is not limited thereto, but may be variously modified and altered by those skilled in the art to which the present disclosure pertains without departing from the spirit and scope of the present disclosure claimed in the following claims.
Therefore, the exemplary embodiments of the present disclosure are provided to explain the spirit and scope of the present disclosure, but not to limit them, so that the spirit and scope of the present disclosure is not limited by the embodiments. The scope of the present disclosure should be construed on the basis of the accompanying claims, and all the technical ideas within the scope equivalent to the claims should be included in the scope of the present disclosure.
Claims
1. A lane recognition device, comprising:
- a camera configured to capture an image in front of a vehicle; and
- a controller configured to recognize a lane based on a curved point of the lane obtained from the image in front of the vehicle.
2. The lane recognition device of claim 1, wherein the controller is configured to:
- detect the lane from the image in front of the vehicle;
- generate a plurality of lane equations based on the curved point of the lane; and
- recognize the lane on a road based on the plurality of lane equations.
3. The lane recognition device of claim 2, wherein the controller is further configured to:
- convert the detected lane from a camera coordinate system to a road coordinate system.
4. The lane recognition device of claim 2, wherein the controller is further configured to:
- detect the lane in a setting interval of a minimum distance closest to the vehicle to a maximum distance most distant from the vehicle in the image in front of the vehicle;
- approximate the detected lane to a lane equation; and
- detect the curved point, when an error between the lane and the lane equation is greater than a first threshold.
5. The lane recognition device of claim 4, wherein the error is formed by the lane and the lane equation.
6. The lane recognition device of claim 4, wherein the controller is further configured to:
- generate a first error graph between the lane and the lane equation in a process of performing curve fitting of a lane set configuring the lane in ascending order of x values;
- generate a second error graph between the lane and the lane equation in a process of performing curve fitting of the lane set in descending order of the x values; and
- set a point where the first error graph and the second error graph are met as the curved point.
7. The lane recognition device of claim 6, wherein the controller is further configured to:
- generate a first lane equation by performing curve fitting of a lane set in an interval of the minimum distance to the curved point in the ascending order of the x values; and
- generate a second lane equation by performing curve fitting of a lane set in an interval of the maximum distance to the curved point in the descending order of the x values.
8. The lane recognition device of claim 7, wherein the controller is configured to:
- recognize the lane based on the first lane equation and the second lane equation.
9. The lane recognition device of claim 4, further comprising:
- a radar configured to detect a location and a relative speed of a preceding vehicle which is located in front of the vehicle.
10. The lane recognition device of claim 9, wherein the controller is further configured to:
- calculate a driving trajectory based on the location and the relative speed of the preceding vehicle;
- perform curve fitting of a set configuring the driving trajectory; and
- reduce the first threshold, when a curvature rate of the driving trajectory is greater than a second threshold.
11. The lane recognition device of claim 4, further comprising:
- a communicator configured to obtain a driving trajectory of the road on which the vehicle is currently traveling from a driving trajectory server.
12. The lane recognition device of claim 11, wherein the controller is configured to:
- perform curve fitting of a set configuring the driving trajectory obtained via the communicator; and
- reduce the first threshold, when a curvature rate of the driving trajectory is greater than a second threshold.
13. A lane recognition method, comprising:
- capturing, by a camera of a vehicle; an image in front of the vehicle; and
- recognizing, a controller of the vehicle, a lane based on a curved point of the lane obtained from the image in front of the vehicle.
14. The lane recognition method of claim 13, wherein the recognizing of the lane includes:
- detecting the lane from the image;
- generating a plurality of lane equations based on the curved point of the lane; and
- recognizing the lane on a road based on the plurality of lane equations.
15. The lane recognition method of claim 14, wherein the detecting of the lane includes:
- converting the detected lane from a camera coordinate system to a road coordinate system.
16. The lane recognition method of claim 14, wherein the calculating of the plurality of equations includes:
- detecting the lane in a setting interval of a minimum distance closest to the vehicle to a maximum distance most distant from the vehicle in the image in front of the vehicle;
- approximating the lane to a lane equation; and
- detecting the curved point, when an error between the lane and the lane equation is greater than a first threshold.
17. The lane recognition method of claim 16, wherein the detecting of the curved point includes:
- generating a first error graph between the lane and the lane equation in a process of performing curve fitting of a lane set configuring the detected lane in ascending order of x values;
- generating a second error graph between the lane and the lane equation in a process of performing curve fitting of the lane set in descending order of the x values; and
- setting a point where the first error graph and the second error graph are met as the curved point.
18. The lane recognition method of claim 17, wherein the calculating of the plurality of lane equations includes:
- generating a first lane equation by performing curve fitting of a lane set in an interval of the minimum distance to the curved point in the ascending order of the x values; and
- generating a second lane equation by performing curve fitting of a lane set in an interval of the maximum distance to the curved point in the descending order of the x values.
19. The lane recognition method of claim 16, wherein the calculating of the plurality of equations further includes:
- calculating a driving trajectory based on a location and a relative speed of a preceding vehicle;
- performing curve fitting of a set configuring the driving trajectory; and
- reducing the first threshold, when a curvature rate of the driving trajectory is greater than a second threshold.
20. The lane recognition device of claim 16, wherein the calculating of the plurality of equations further includes:
- obtaining a driving trajectory of the road on which the vehicle is currently traveling from a driving trajectory server;
- performing curve fitting of a set configuring the driving trajectory; and
- reducing the first threshold, when a curvature rate of the driving trajectory is greater than a second threshold.
Type: Application
Filed: Jun 10, 2019
Publication Date: Jul 23, 2020
Patent Grant number: 11164011
Inventor: Kyung Deuk MIN (Hwaseong-si)
Application Number: 16/436,027