PREDICTIVE SEMI-AUTONOMOUS VEHICLE NAVIGATION SYSTEM

Abstract

An active safety framework performs trajectory planning, threat assessment, and semi-autonomous control of passenger vehicles in a unified, optimal fashion. The vehicle navigation task is formulated as a constrained optimal control problem. A predictive, model-based controller iteratively plans an optimal or best-case vehicle trajectory through the constrained corridor. This best-case scenario is used to establish the minimum threat posed to the vehicle given its current state, current and past driver inputs/performance, and environmental conditions. Based on this threat assessment, the level of controller intervention required to prevent collisions or instability is calculated and driver/controller inputs are scaled accordingly. This approach minimizes controller intervention while ensuring that the vehicle does not depart from a traversable corridor. It also provides a unified architecture into which various vehicle models, actuation modes, trajectory-planning objectives, driver preferences, and levels of autonomy can be seamlessly integrated without changing the underlying controller structure.

Description

RELATED DOCUMENT

Priority is hereby claimed to U.S. Provisional application Ser. No. 61/209,250, entitled PREDICTIVE SEMI-AUTONOMOUS VEHICLE NAVIGATION SYSTEM, in the names of Sterling J. Anderson, Steven C. Peters and Karl D. Iagnemma, filed on Mar. 5, 2009, which is hereby fully incorporated herein by reference.

BRIEF DESCRIPTION OF THE FIGURES OF THE DRAWING

FIG. 1 is a block diagram illustrating basic framework operation.

FIG. 2 graphically shows an example of various potential intervention laws based on threat metric calculation.

FIG. 3 graphically shows an obstacle avoidance scenario illustrating different stages of intervention for an inattentive driver.

FIG. 4 shows, in flowchart form, a basic algorithm logic flow with possible considerations at each step.

FIG. 5 shows, graphically, a simulated test illustrating system response when driver fails to navigate a curve in the road. K represents the proportion of control authority given to autonomous system, with the driver allowed the remaining (1-K).

FIG. 6 shows, graphically, a simulated test illustrating system response to an erroneous driver swerve, where K represents the proportion of control authority given to autonomous system, with the driver allowed the remaining (1-K).

FIG. 7 shows, graphically, a simulated test illustrating system response when driver fails to anticipate/avoid obstacle. K represents the proportion of control authority given to autonomous system, with the driver allowed the remaining (1-K).

DETAILED DESCRIPTION

Inventions described herein relate to a unified framework for performing threat assessment and semi-autonomous vehicle navigation and control while allowing for adaptable and configurable intervention laws and configurable control inputs.

Automotive active safety systems are concerned with preventing accidents through the introduction of various computer-controlled actuation methods to improve driver braking and steering performance. Current active safety systems include yaw stability control, roll stability control, traction control, and antilock braking, among others. While these systems reduce accident frequency, they are fundamentally reactive in nature: their intervention is based on current vehicle (and, possibly, road surface) conditions. Because they do not utilize 1) sensory information related to the vehicle surroundings or 2) a prediction of the vehicle's path through its surroundings, they have limited ability to assess the threat of impending accidents, and thus cannot exert corrective actions to avoid them.

Active navigation systems, such as the one described here, aim to avoid accidents by utilizing sensory information related to the vehicle surroundings and a prediction of a safe vehicle trajectory through those surroundings to exert appropriate actuator effort to avoid impending accidents. Sensory information would include data related to nearby vehicles, pedestrians, road edges, and other salient features to assess accident threat.

Such navigation systems ideally operate only during instances of significant threat: it should give a driver full control of the vehicle in low threat situations but apply appropriate levels of computer-controlled actuator effort during high threat situations. An active navigation system can therefore be termed semi-autonomous, since it must allow for human-controlled, computer-controlled, and shared human/computer vehicle operation. Such a system should be as unobtrusive to the driver as possible (i.e. it should intervene only as much as is minimally required to avoid an impending accident).

The semi-autonomous active navigation system described here satisfies all of the above requirements and desired characteristics. Further, it provides a framework into which various distinct sensing and actuation modes can be easily incorporated. The system's method for threat assessment and computer-controlled intervention can potentially be modified in real time based on the scenario, environmental conditions, driver preference, or past driver performance. FIG. 1 shows, schematically, in block diagram form, a basic framework operation.

This semi-autonomous vehicle navigation system predicts an optimal (with respect to a pre-defined, configurable set of criteria) vehicle trajectory from the current position through a finite time horizon given a model of the environment, a model of the vehicle, the vehicle's current state, and a corresponding optimal set of control inputs (also calculated by the system). The environment model can be based on a priori known information (e.g. from maps) and/or information gathered by real time sensors, such as on-vehicle sensors (e.g. cameras and laser rangefinders), and can include information related to the position of road edges, static obstacles (e.g. trees, road-side signs), and dynamic obstacles (e.g. other vehicles, pedestrians). The vehicle model is user-defined and can be of varying complexity and fidelity. The real-time sensors may also be mounted in the environment and communicate with the control system on the vehicle.

The predicted safe vehicle trajectory (and associated control inputs to yield such a trajectory) is found such that it satisfies a configurable set of trajectory requirements, including, for example, that the vehicle position remain within a safe driving corridor, that the vehicle sideslip angle not exceed the safe limit of vehicle handling, that tire friction forces not exceed a surface friction-limited value, and others. The control inputs can be associated with one or multiple actuators, such as active steering, active braking, and others. The predicted vehicle trajectory and associated control inputs may be computed via constrained optimal control, which leverages efficient optimization methods and constraint-handling capabilities.

At successive discrete sampling instants, the predicted vehicle trajectory and control inputs are analyzed to assess the threat to the vehicle by computing a configurable metric, such as the maximum lateral acceleration, sideslip angle, or roll angle over the trajectory, the minimum proximity to obstacles, or others. The control authority exerted by the system is then determined as a function of this computed threat: generally speaking, if the threat metric value is low, the control system intervention is low (i.e. the driver commands the vehicle with little or no computer-controlled intervention); if the threat metric value is high, the control system intervention is high. The form of the intervention law modulating this control system authority is configurable and can differ for different actuators (i.e. a vehicle with both active steering and braking can have distinct intervention laws defined for the steering actuator and the braking actuators). The intervention law can also be defined to adapt to driver performance based on an assessment of driver skill, and/or to include considerations for driver preference, environmental conditions, previous threat metric values, previous control inputs, and other factors. FIG. 2 shows, schematically, examples of various potential intervention laws, showing, from top to bottom, linear, smooth and threshold-shaped intervention laws that depend only on predicted threat. The vertical axis represents the degree of control authority given to the active navigation and control system while the horizontal axis represents the predicted threat, with cause for intervention increasing from left to right.

In the system described above, as the threat metric value increases, indicating that the predicted vehicle trajectory will near a pre-defined critical vehicle state(s) (such as spatial location, lateral acceleration, or tire friction saturation), the control system begins to assume control authority to preempt an unsafe maneuver. As the threat metric decreases, the controller's authority phases out. In this manner, the system can said to be semi-autonomous.

Note that in extreme cases, when the driver does not perform an appropriate corrective action, it is conceivable that a required hazard avoidance maneuver will reach vehicle handling limits. To account for such scenarios, the intervention law can be designed such that it assumes full authority by the time the predicted safe trajectory reaches the limit of any pre-defined critical vehicle states. This corresponds to a situation where only an optimal set of inputs would result in a safe vehicle trajectory.

FIG. 3 shows schematically an obstacle avoidance scenario illustrating different stages of intervention for an inattentive driver. FIG. 4 shows, schematically, in flow chart form, a basic flow of logic performed by a controller of an invention hereof, with possible considerations at each step.

An initial step calculates an optimal set of control inputs and corresponding vehicle trajectory. Considerations for this step include, for example, (but are not limited to) the vehicle dynamics, current state of the vehicle and environment, terrain and environmental disturbances, available actuation, trajectory objectives, safety limits, and driver inputs.

A next step is to assess the predicted threat to the vehicle. Considerations for this step include characteristics of the optimal path and associated control input, safety limits and driver inputs. A next step is to calculate control authority gains, with a major consideration at this stage being the desired intervention characteristic. The next step is to implement the scaled control for the current time.

Simulation experiments have been conducted. FIG. 5 shows, graphically, the results of a simulated test illustrating system response when a driver fails to navigate a curve in the road, shown by a light gray line. The trajectory that the driver would have followed without assistance is shown dashed. With assistance, it is shown solid black. Note that in this embodiment of the invention, K represents proportion of control authority given to the autonomous system, with the driver allowed the remaining (1-K). The middle graph shows the steer inputs, with the dashed line corresponding to the driver and the solid curve corresponding to the control system. The lower graph shows the control authority given to the autonomous system, in this case, steering, with the degree varying with distance (x) along the horizontal scale.

FIG. 6 shows, graphically, the results of a simulated test illustrating the system response to an erroneous driver swerve. Again, K represents proportion of control authority given to autonomous system, with the driver allowed the remaining (1-K). The same line types as above correspond to the driver without assistance (gray dashed) and with assistance (solid line). The safe roadway is shown in light gray solid lines in the upper graph. Distance is shown along the horizontal scale.

FIG. 7 shows, graphically, a simulated test illustrating system response when a driver fails to anticipate/avoid an obstacle. Again, K represents the proportion of control authority given to autonomous system. The obstacle is simulated by a jog in the light gray line that represents the safe roadway. The only inputs used in this simulation are, again, steering of the driver and the autonomous system.

Significant advantages stem from the predictive nature of this solution. In addition to considering past and current vehicle and driver actions to assess threat and determine control authority, the current solution predicts a future vehicle trajectory and associated threat, and uses this prediction to schedule control authority.

This predictive nature also allows for a more accurate assessment of threat than is otherwise possible. While other threat assessment metrics rely on highly simplified physics-based calculations, the metrics used in the current solution can derive from sophisticated vehicle and environmental models. These models yield more accurate threat assessments by considering the effects of terrain conditions, environmental disturbances, and physical limitations of vehicle actuators. These models can also assess threat for more complex vehicle trajectories than is possible with simplified models.

Finally, this system provides improved modularity and adaptability when compared to previous solutions. Its underlying control framework can accommodate multiple actuation modes and vehicle models, allowing for ready application of the system to various vehicle types and actuator configurations. The system's intervention law is also readily adapted (i.e. it can change over time based on an assessment of driver skill, driver preference, environmental conditions, previous threat metric values, previous control inputs, and other factors). These adaptations can be performed either statically or dynamically.

SUMMARY

An important aspect of inventions disclosed herein is a method for generating a set of machine control inputs for semi-autonomously controlling a vehicle operating in an environment, with a variable degree of human operator control relative to the degree of machine control. The method comprises the steps of: predicting an optimal vehicle trajectory from a current position through a time horizon; assessing a predicted threat to the vehicle and generating a corresponding threat metric; based on the threat metric, generating at least one control authority gain; and generating at least one machine control optimal input; and generating at least one machine control scaled input, based on the machine control optimal input and the control authority gain. In this manner, the degree of machine control of the vehicle relative to the degree of human operator control of the vehicle varies depending on the control authority gain.

With a closely related method, the step of predicting an optimal vehicle trajectory is based on: a model of the environment; a model of the vehicle; the vehicle's current state; driver inputs; and a corresponding optimal set of control inputs.

For another important related method, the step of assessing a predicted threat is based on: characteristics of optimal vehicle path and associated control input; environmentally imposed safety constraints; and driver inputs.

Still another important aspect has the step of generating at least one machine control scaled input being based on an intervention characteristic. In such a case, the intervention characteristic may be chosen from the group consisting of: a linear function of current and past predicted threat, and current and past control input; and a nonlinear function of current and past predicted threat, and current and past control input.

The environmental model may be based on a priori known information.

Or, the environmental model may be based on information gathered by real-time sensors.

Another interesting embodiment has the threat metric being at least one metric selected from the group consisting of: maximum lateral acceleration, sideslip angle, roll angle over the trajectory and a minimum proximity to obstacles.

Another aspect of an invention hereof has a threat metric being at least one metric selected from the group consisting of: characteristics of the optimal vehicle path and control input, including predicted vehicle states such as lateral acceleration, vehicle sideslip angle, tire sideslip angle, road friction utilization, roll angle, pitch angle, past and present driver performance, environmentally-imposed safety constraints, and proximity to hazards. For a specific aspect, the threat metric may be at least one metric based on one the group consisting of: average, maximum, minimum, and RMS norms of a predicted vehicle state. In such a case, the predicted vehicle state may be selected from the group consisting of: lateral acceleration, vehicle sideslip angle, tire sideslip angle, road friction utilization, roll angle, pitch angle, driver inputs, and proximity to hazards.

For a related embodiment, the optimal vehicle trajectory and associated optimal control inputs can be computed by constrained optimal control.

The vehicle may be an automotive vehicle, with at least one sensor generating data related to at least one of the factors in the group consisting of: nearby vehicles, pedestrians, road edges, roadway hazards, road surface friction and other environmental characteristics.

With most such methods, the control authority gain may be such that if the threat metric value is low, the control system intervention is low and thus, the human operator controls the vehicle with minimal computer-controlled intervention). Conversely, if the threat metric value is high, the control system intervention is high and thus, the human operator controls the vehicle with significant computer controlled intervention.

For a very useful embodiment, the vehicle may comprise an automotive vehicle, where at least one machine control scaled input is selected from the group consisting of: steering, braking and acceleration.

For many related embodiments, the optimal set of machine control inputs used in the step of predicting an optimal vehicle trajectory may comprise machine control inputs having been generated by the method for generating a set of automated control inputs.

Another aspect of inventions disclosed herein is an apparatus for generating a set of machine control inputs, thereby controlling a vehicle operating in an environment, with a variable degree of human operator control and a variable degree of machine control. The apparatus comprises: a. means for predicting an optimal safe vehicle trajectory from a current position through a time horizon, based on: a model of the environment; a model of the vehicle; the vehicle's current state; driver inputs; and a corresponding optimal set of control inputs. This aspect of the inventions further includes: b. means for assessing a predicted threat to the vehicle and generating a corresponding threat metric; c. a machine controller that generates at least one machine control optimal input; d. means for generating at least one control authority gain based on the threat metric; e. means for generating at least one machine control scaled input based on the at least one machine control optimal input and the control authority gain; f. means for generating a scaled human operator input, based on a human operator command, and the control authority, whereby the human operator scaled input is also based on the control authority gain, inversely to the degree that the machine control scaled input is based on the at least one machine control optimal input; and g. an input combiner, which combines the human operator scaled input and the machine control scaled input to an actuator that actuates a system of the vehicle.

Another basic aspect of inventions hereof is an automotive vehicle having a chassis, wheels, a power plant, a body, and a control apparatus, the control apparatus generating a set of machine control inputs, thereby controlling the vehicle while operating in an environment, with a variable degree of human operator control and a variable degree of machine control. The control apparatus comprises: a. means for predicting an optimal safe vehicle trajectory from a current position through a finite time horizon based on: a model of the environment; a model of the vehicle; the vehicle's current state; driver inputs; and a corresponding optimal set of control inputs. The control apparatus further comprises: b. means for assessing a predicted threat to the vehicle and generating a corresponding threat metric; c. a machine controller that generates at least one optimal machine control input; d. means for generating at least one control authority gain based on the threat metric; e. means for generating at least one machine control scaled input based on the at least one machine control optimal input and the control authority gain; f. means for generating a scaled human operator input, based on a human operator command, and the control authority, whereby the human operator scaled input is also based on the control authority gain, inversely to the degree that the machine control scaled input is based on the at least one optimal control input; and g. an input combiner, which combines the human operator scaled input and the machine control scaled input to an actuator that actuates a system of the vehicle.

This disclosure describes and discloses more than one invention. The inventions are set forth in the claims of this and related documents, not only as filed, but also as developed during prosecution of any patent application based on this disclosure. The inventors intend to claim all of the various inventions to the limits permitted by the prior art, as it is subsequently determined to be. No feature described herein is essential to each invention disclosed herein. Thus, the inventors intend that no features described herein, but not claimed in any particular claim of any patent based on this disclosure, should be incorporated into any such claim.

Some assemblies of hardware, or groups of steps, are referred to herein as an invention. However, this is not an admission that any such assemblies or groups are necessarily patentably distinct inventions, particularly as contemplated by laws and regulations regarding the number of inventions that will be examined in one patent application, or unity of invention. It is intended to be a short way of saying an embodiment of an invention.

An abstract is submitted herewith. It is emphasized that this abstract is being provided to comply with the rule requiring an abstract that will allow examiners and other searchers to quickly ascertain the subject matter of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims, as promised by the Patent Office's rule.

The foregoing discussion should be understood as illustrative and should not be considered to be limiting in any sense. While the inventions have been particularly shown and described with references to preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the inventions as defined by the claims.

The corresponding structures, materials, acts and equivalents of all means or step plus function elements in the claims below are intended to include any structure, material, or acts for performing the functions in combination with other claimed elements as specifically claimed.

Claims

1. A method for generating a set of machine control inputs for semi-autonomously controlling a vehicle operating in an environment, with a variable degree of human operator control relative to the degree of machine control, the method comprising the steps of:

a. predicting an optimal vehicle trajectory from a current position through a time horizon:

b. assessing a predicted threat to the vehicle and generating a corresponding threat metric;

c. based on the threat metric, generating at least one control authority gain; and

d. generating at least one machine control optimal input; and

e. generating at least one machine control scaled input, based on the machine control optimal input and the control authority gain;

whereby the degree of machine control of the vehicle relative to the degree of human operator control of the vehicle varies depending on the control authority gain.

2. The method of claim 1, the step of predicting an optimal vehicle trajectory being based on:

a. a model of the environment;

b. a model of the vehicle;

c. the vehicle's current state;

d. driver inputs; and

e. a corresponding optimal set of control inputs;

3. The method of claim 1, the step of assessing a predicted threat being based on:

a. characteristics of optimal vehicle path and associated control input;

b. environmentally imposed safety constraints; and

c. and driver inputs.

4. The method of claim 1, the step of generating at least one machine control scaled input being based on an intervention characteristic.

5. The method of claim 4, the intervention characteristic being chosen from the group consisting of a linear function of current and past predicted threat, and current and past control input; and a nonlinear function of current and past predicted threat, and current and past control input.

6. The method of claim 1, the environmental model being based on a priori known information.

7. The method of claim 1, the environmental model being based on information gathered by real-time sensors.

8. The method of claim 1, the threat metric being at least one metric selected from the group consisting of:

maximum lateral acceleration, sideslip angle, roll angle over the trajectory and a minimum proximity to obstacles.

9. The method of claim 1, the optimal vehicle trajectory and associated optimal control inputs being computed by constrained optimal control.

10. The method of claim 3, the vehicle comprising an automotive vehicle, with at least one sensor generating data related to at least one of the factors in the group consisting of: nearby vehicles, pedestrians, road edges, roadway hazards, road surface friction and other environmental characteristics.

11. The method of claim 1, control authority gain being such that if the threat metric value is:

a. low, the control system intervention is low and thus, the human operator controls the vehicle with minimal computer-controlled intervention); and

b. high, the control system intervention is high and thus, the human operator controls the vehicle with significant computer controlled intervention.

12. The method of claim 1, the vehicle comprising an automotive vehicle, at least one machine control scaled input being selected from the group consisting of: steering, braking and acceleration.

13. The method of claim 1, the optimal set of machine control inputs used in the step of predicting an optimal vehicle trajectory comprising machine control inputs having been generated by the method for generating a set of automated control inputs.

14. An apparatus for use of generating a set of machine control inputs thereby controlling a vehicle operating in an environment, with a variable degree of human operator control and a variable degree of machine control, the apparatus comprising:

a. means for predicting an optimal safe vehicle trajectory from a current position through a time horizon based on; i. a model of the environment; ii. a model of the vehicle; iii. the vehicle's current state; iv. driver inputs; and v. a corresponding optimal set of control inputs;

b. means for assessing a predicted threat to the vehicle and generating a corresponding threat metric;

c. a machine controller that generates at least one machine control optimal input;

d. means for generating at least one control authority gain based on the threat metric;

e. means for generating at least one machine control scaled input based on the at least one machine control optimal input and the control authority gain;

f. means for generating a scaled human operator input, based on a human operator command, and the control authority, whereby the human operator scaled input is also based on the control authority gain, inversely to the degree that the machine control scaled input is based on the at least one machine control optimal input; and

g. an input combiner, which combines the human operator scaled input and the machine control scaled input to an actuator that actuates a system of the vehicle.

15. An automotive vehicle having a chassis, wheels, a power plant, a body, and a control apparatus, the control apparatus generating a set of machine control inputs thereby controlling the vehicle while operating in an environment, with a variable degree of human operator control and a variable degree of machine control the control apparatus comprising:

a. means for predicting an optimal safe vehicle trajectory from a current position through a time horizon based on; i. a model of the environment; ii. a model of the vehicle; iii. the vehicle's current state; iv. driver inputs; and v. a corresponding optimal set of control inputs;

b. means for assessing a predicted threat to the vehicle and generating a corresponding threat metric;

c. a machine controller that generates at least one optimal machine control input;

d. means for generating at least one control authority gain based on the threat metric;

e. means for generating at least one machine control scaled input based on the at least one machine control optimal input and the control authority gain;

f. means for generating a scaled human operator input, based on a human operator command, and the control authority, whereby the human operator scaled input is also based on the control authority gain, inversely to the degree that the machine control scaled input is based on the at least one optimal control input; and

g. an input combiner, which combines the human operator scaled input and the machine control scaled input to an actuator that actuates a system of the vehicle.

16. The method of claim 1, the threat metric being at least one metric selected from the group consisting of:

characteristics of the optimal vehicle path and control input, including predicted vehicle states of lateral acceleration, vehicle sideslip angle, tire sideslip angle, road friction utilization, roll angle, pitch angle, past and present driver performance, environmentally-imposed safety constraints, and proximity to hazards.

17. The method of claim 16, the threat metric being based on at least one metric selected from the group consisting of:

average, maximum, minimum, and RMS norms of a predicted vehicle state.

18. The method of claim 17, the predicted vehicle state being selected from the group consisting of: lateral acceleration, vehicle sideslip angle, tire sideslip angle, road friction utilization, roll angle, pitch angle, driver inputs, and proximity to hazards.