METHOD AND APPARATUS FOR SIMULTANEOUS LOCALIZATION AND MAPPING OF MOBILE ROBOT ENVIRONMENT
Techniques that optimize performance of simultaneous localization and mapping (SLAM) processes for mobile devices, typically a mobile robot. In one embodiment, erroneous particles are introduced to the particle filtering process of localization. Monitoring the weights of the erroneous particles relative to the particles maintained for SLAM provides a verification that the robot is localized and detection that it is no longer localized. In another embodiment, cell-based grid mapping of a mobile robot's environment also monitors cells for changes in their probability of occupancy. Cells with a changing occupancy probability are marked as dynamic and updating of such cells to the map is suspended or modified until their individual occupancy probabilities have stabilized. In another embodiment, mapping is suspended when it is determined that the device is acquiring data regarding its physical environment in such a way that use of the data for mapping will incorporate distortions into the map, as for example when the robotic device is tilted.
Latest NEATO ROBOTICS, INC. Patents:
The present application claims the benefit of co-pending U.S. provisional application Ser. No. 61/238,597, filed Aug. 31, 2009, entitled “Computation Optimization Techniques for Simultaneous Localization and Mapping,” the disclosure of which is incorporated by reference herein in its entirety.
BACKGROUNDAspects of the present invention relate to mobile robots, and more particularly to the mapping of environments in which mobile robots operate, to facilitate movement of mobile robots within those environments.
As a system that enables a mobile robot to map its environment and maintain working data of its position within that map, simultaneous localization and mapping (SLAM) is both accurate and versatile. Its reliability and suitability for a variety of applications make it a useful element for imparting a robot with some level of autonomy.
Typically, however, SLAM techniques tend to be computationally intensive and thus their efficient execution often requires a level of processing power and memory capacity that may not be cost effective for some consumer product applications.
For those facing the low-cost production targets necessary for competition in the consumer market, it is unlikely that an economic hardware environment would include processing and memory capacities capable of supporting adequately a robust SLAM system. It therefore is imperative that developers seek ways to facilitate efficient execution of the core SLAM algorithms within the limits of the computational capacities they have. Generally, such optimization schemes would seek to use processing power and system bandwidth judiciously, which might mean simplifying some of the SLAM algorithms in ways that do not critically compromise their performance, or reducing input data size or bandwidth.
SUMMARY OF THE INVENTIONFour concepts are outlined herein, each intended to enable a SLAM system to maintain efficiency when it is operating on a platform that provides limited processor power and/or memory capacity. Some of these optimization methods may reside entirely in software, or may require some element of hardware support to function properly.
Localization requires regularly updating a robot's pose (position and angle) within its environment. The frequency with which this is done can affect overall system performance, depending on how often data must be processed as a result of an update operation. Minimizing computational load is essential to providing a SLAM system that can function effectively in a low-cost hardware environment.
According to one feature of the invention, computational load may be reduced by eliminating robot position updates when it appears that the robot has become delocalized, in which case the updates likely would be erroneous anyway.
The method may operate as follows:
-
- 1) Erroneous position and inclination particles may be introduced to the set of tracking particles. The erroneous particles, also referred to later as verification particles, may be selected in a way that they likely will not introduce additional error into the current estimate of the robot's position and inclination.
- 2) Typically, erroneous particles have low weights, which may correspond generally to their low probability of accurately representing the robot's current position. If the erroneous particles have weights that are not uniformly low, but rather may be a distribution or some combination of low and high weights, then this may imply that the robot has become delocalized.
- 3) If it is determined that the robot likely is delocalized, then updating its position within the map of its surroundings may be suspended until the weights of the erroneous particles return to a more uniform distribution of low values.
There are precautionary reasons why this procedure is implemented in a SLAM system and it may afford other advantages beyond computational load reduction. Suspension of mapping when delocalization is detected may avoid corrupting the map. Also, once delocalization is detected, additional actions can be enabled to improve the likelihood that the robot will re-localize, such as increasing the number of particles in the set or employing looser error models. Depending on the severity of the delocalization, other actions might be taken aside from those that are related to recovery. For example, the robot might stop or restart its run.
Example Determining Delocalization Through Introduction of Erroneous ParticlesA typical approach to localization under a SLAM scheme might include the following steps:
1) For each particle:
-
- a) Apply an ideal motion model (e.g., odometry).
- b) Apply position and angle (x,y,θ) adjustments drawn from error model distributions.
- c) Evaluate with respect to the current map to compute weight.
2) Resample particles proportional to computed weights.
A typical localization iteration based on the above process might yield the particle weight distribution illustrated in
In
It is worth noting that the weight scale (the vertical axis in the graph) may be highly dependent on environmental conditions such as distance from walls, number of valid distance readings from a spatial sensor such as a laser rangefinder, etc. An approach to determining delocalization via the introduction of erroneous particles generally should be independent of environmental conditions.
The goal of introducing erroneous particles is to identify when the particles with higher probability of representing the robot's pose are not much better than particles with the lowest probability of representing the robot's pose. In such a circumstance, the implication is that most or all potential poses are bad, and therefore the robot has little or no reliable information regarding its actual whereabouts within its environment. By definition, the robot is delocalized.
The process of assessing the state of localization involves introducing additional test particles whose pose is deliberately erroneous in order to set a baseline weight for comparison to better particles.
It is often observed that particle evaluation is most sensitive to angular errors. Small changes in robot angle, for example, can translate to large errors in distance measurements as the distance from the robot to an object in its surrounding environment increases. Large angular errors can have similar distributions of laser readings in terms of distance, but they may dramatically reduce the overall weight of the full particle set.
Typically, the particles representing candidate location angles with the highest weights are fairly close to an ideal motion model. Recognizing this, a generally effective approach to delocalization detection is to introduce erroneous particles at the center of the ideal motion model with large offsets to the angle (e.g., ±30°, 40°, 50°, 60°, etc.).
If the robot is properly localized, the erroneous particles will reside relatively close together at the end of the sorted distribution curve that contains the lowest weighted particles, as shown in
In
If the robot is delocalized, many normal particles will have low weights, and many of these are likely to have weights lower than some of the erroneous or verification particles, as seen in
In
The actual determination of delocalization can be done in any of a variety of ways, including by examining the mean index value of the erroneous (verification) particles. In a localized condition, most or all of the erroneous particles will reside relatively close together at the bottom of the index, since they generally will have the lowest weights. Averaging the indices of the erroneous particles in a localized case will yield a large number relative to the size of the total set of particles, including both erroneous and non-erroneous particles.
In a delocalized state, however, the erroneous particles are scattered through the distribution curve, and an indexing of particles in order of their weight will yield a set of erroneous particles whose averaged index is not necessarily high with respect to the size of the total set of particles. Generally, an average of verification particle indices that remains constant and high in value with respect to total particle set size reflects a localized condition. An average that falls in value or begins to fluctuate in value may indicate a delocalized condition.
Both of these states, localized and delocalized, are depicted in the plots of the averaged verification particle data in
At a point on the graph between 800 and 1000 localization iterations the data begins to fluctuate greatly. The lack of consistency in the average and the range of its variability are indicative of a delocalized condition. Unlike the previous, large delocalization, the robot likely was unable to recover from this delocalization as indicated by the data's continuing instability through the end of the data set.
Determining that the robot has delocalized relies on comparing the averaged erroneous particle index to a threshold number. The threshold number can be decided a priori during coding, but it is typically beneficial to include some hysteresis in the evaluation of whether a robot is localized. For example, looking at the latter portion of the data set illustrated in
One of the challenges confronting a robot engaged in creation and update of maps of its surroundings is the potential mix of static and dynamic elements within its surroundings. While it is generally expected that most of a robot's surroundings will remain fixed, a robot should be prepared to function within an environment in which people, pets, etc. may be moving.
Newly encountered, unmapped space may contain a mix of dynamic and static elements. Making a distinction between the robot's identification of potentially dynamic areas of the map and those that are static is essential for building useful and accurate maps for the robot to use.
In an embodiment, the issue of distinguishing between static (permanent) elements of the robot's surroundings and dynamic (transient) elements may be addressed in the following way:
-
- 1) The robot may create an abstraction of its environment (a map) within a grid-space of cells available in memory, each cell containing a number that indicates a relative probability of whether the space within the cell is empty or occupied. These values may range from, for example, zero (empty) to 254 (occupied), with an initial condition value within every cell of 127 (i.e., a value in the middle of the spectrum).
- 2) A spatial sensor, most conveniently a laser rangefinder, may scan the robot's surroundings, measuring distances to boundaries and other objects. This data stream may provide the base information from which the robot can determine the probability that a cell is occupied or not. For example, if the spatial sensor measures a distance to a wall, the occupancy probability that the cell on the robot-generated map corresponding to that point along the wall is occupied increases while the occupancy probability for all the cells along the measurement vector between the robot and the wall decreases (because the wall was the first object detected). With repeated measurement from the spatial sensor, the probabilities may become more certain.
- 3) If a cell currently identified as empty has an occupancy probability that is changing (e.g., appearing suddenly to be occupied), it may signify a potentially dynamic area of the map.
- 4) If such cells are detected, they may be marked so as to not be updated with regard to their likelihood of containing an obstacle while they are dynamic. Similarly, this also can extend to an arbitrary zone surrounding these cells.
Accurate delineation of a robot's surroundings as part of mapping and localization requires maintaining the orientations of the sensors generating spatial data in congruence with the presiding surfaces of the surrounding geometry. For a robot operating inside a building or similar enclosure, this means that a sensor collecting information in two dimensions would preferably maintain its plane of detection as parallel to the floor since the floor would define the dominant plane of motion available to a robot traversing it.
Because floors may have areas of uneven surface or surface discontinuities, or because objects resting on the floor may introduce non-uniformities in a robot's available travel surface, it is possible that a sensor collecting spatial data may not maintain consistent orientation with the presiding surfaces of the surrounding geometry, which can lead to erroneous delineation of the robot's surroundings.
Often, wheel slip accompanies tilt when a robot traverses a substantive irregularity in a floor surface. This can be particularly problematic if it occurs when the robot is collecting its first data on a new area (e.g., when the robot has turned a corner into an unmapped space) since the distorted image may be incorporated into the map.
For a robot using the continuous generation of spatial boundary information to provide updates to a map, erroneous data generated during a tilt event can propagate into mapping or localization algorithms. The potential results may include some degree of mapping corruption, which frequently can lead to delocalization.
Consequently, it is important to provide a strategy to identify and address tilt conditions during normal operation, and two approaches to same are described below. These approaches are designed such that they can be used separately or together in potential reinforcement.
3. Tilt as Detected and Addressed in SoftwareTypically, dynamic areas created by people, pets or objects moved or in use by a person will present a dynamic area to mark, one that usually is limited in its footprint. However, if the dynamic area is spread along a relatively wide area, then this may represent a different scenario. For example, if a map boundary area shifts suddenly or moves in a way that many, possibly contiguous cells are tagged as active, then it may be likely that the robot has tilted. In such a case, the spatial sensor's detection plane may be angled such that a portion of the floor near the robot is read as a boundary, as indicated in the example described earlier. When the robot identifies that a dynamic area involves an area larger than would be created by people, pets or moving objects in relative proportion with the former, then the updating of the map may be suspended.
It should be noted that instructions to suspend or modify the use of generated data for mapping need not come solely from the sensing unit or from within the processing apparatus. These respective features of the system depicted in
Detection of motion may rely on spatial scanning done by, for example, a laser rangefinder, which may continuously scan a robot's surroundings. When scanning indicates that consecutive distance readings show “dynamic” movement, the spatial distance represented by an aggregate distance, or by a distance differential, may be compared to a pre-defined threshold value. If the difference between the first to the last distance measurement is larger than the threshold, it may be concluded that the robot is tilted.
Detection of tilt in hardware may involve the use of an accelerometer or similar component that may detect changes in the orientation of the component's mounting surface.
With this approach, data generated by the spatial scanner may be supplemented by data regarding changes in orientation. With this latter data set providing contextual verification for the spatial sensor's data, information collected while the tilt-detecting component indicates that the spatial sensor has lost its preferred orientation could be discarded. In a typical embodiment, this data may be discarded before it is processed by any localization or mapping software.
As depicted in
As depicted in
Several features and aspects of the present invention have been illustrated and described in detail with reference to particular embodiments by way of example only, and not by way of limitation. Reference herein to “one embodiment” or “an embodiment” means that a particular feature, structure, operation, or other characteristic described in connection with the embodiment may be included in at least one implementation of the invention. However, the appearance of the phrase “in one embodiment” or “in an embodiment” in various places in the specification does not necessarily refer to the same embodiment. It is envisaged that the ordinarily skilled person could use any or all of the above embodiments individually, or in any compatible combination or permutation. Those of skill in the art will appreciate that alternative implementations and various modifications to the disclosed embodiments are within the scope and contemplation of the present disclosure. Therefore, it is intended that the invention be considered as limited only by the scope of the appended claims.
Claims
1. A mobile robotic system comprising:
- a. a mobile robot; and
- b. a system for controlling movement of the robot, the system comprising; i. a data acquisition system that generates data identifying the robot's physical environment, said data acquisition system having a preferred orientation for operation with respect to the robot's physical environment; ii. processing apparatus, responsive to the data acquisition system, to map or model the robot's physical environment and the data acquisition system's location within the robot's physical environment; and iii. a sensing unit that determines whether the data acquisition system has lost its preferred orientation for operation with respect to the robot's physical environment; iv. wherein the processing apparatus includes apparatus that responds to the sensing unit to suspend or modify the use of data generated by the data acquisition system for mapping or modeling the robot's physical environment when the sensing unit has determined that the data acquisition system has lost its preferred orientation for operation with respect to the robot's physical environment.
2. A mobile robotic system as claimed in claim 1 wherein the data acquisition system comprises a distance-measuring device to identify items in the robot's operating environment.
3. A mobile robotic system as claimed in claim 2 wherein the distance-measuring device comprises light-emitting and light-receiving apparatus to emit light to measure distance based on reflected light.
4. A mobile robotic system as claimed in claim 2 wherein the distance-measuring device comprises a laser rangefinder.
5. A mobile robotic system as claimed in claim 1 wherein the sensing unit comprises an accelerometer.
6. A mobile robotic system as claimed in claim 1 wherein the processing apparatus is physically separate from the mobile robot and is in communication with the mobile robot.
7. A mobile robotic system comprising:
- a. a mobile robot; and
- b. a system for controlling movement of the robot, the system comprising: i. a data acquisition system that generates data identifying the robot's physical environment, said data acquisition system having a preferred orientation for operation with respect to the robot's physical environment; and ii. processing apparatus, responsive to the data acquisition system, to generate a map of the robot's physical environment and the data acquisition system's location within the robot's physical environment; iii. wherein the processing apparatus includes apparatus that responds to the data generated by the data acquisition system to suspend or modify the use of data generated by the data acquisition system when a portion of the map above a threshold limit corresponds to a predetermined shift of one or more elements in the map.
8. A mobile robotic system as claimed in claim 7 wherein the data acquisition system comprises a distance-measuring device to identify items in the robot's physical environment.
9. A mobile robotic system as claimed in claim 8 wherein the distance-measuring device comprises light-emitting and light-receiving apparatus to emit light to measure distance based on reflected light.
10. A mobile robotic system as claimed in claim 8 wherein the distance-measuring device comprises a laser rangefinder.
11. A mobile robotic system as claimed in claim 1 wherein the processing apparatus is physically separate from the mobile robot and is in communication with the mobile robot.
12. A mobile device tracking system comprising:
- a. apparatus that generates a map of the device's physical environment, said generating occurring either concurrently with or prior to tracking the device's position, said generating apparatus updating said map either concurrently with or prior to tracking the device's position; and
- b. a processing apparatus to determine the device's current location within said map by generating particles, each of said particles representing a potential position and/or orientation of the device within its physical environment, wherein a data set of said particles may be generated and maintained iteratively to track changing position of the device within its physical environment, the processing apparatus including: i. apparatus that assigns a weight to each particle, particle weight being a relative measure of the particle's likelihood of accurately representing the robot's position with respect to other particles; ii. apparatus that introduces erroneous particles whose potential positions are selected so that their assigned weights should be uniformly low with respect to the weights of the particles described in (b); and iii. apparatus that compares the weights of the erroneous particles and the weights of the particles described in (b) to determine whether the mobile device has become delocalized when a substantial number of erroneous particles have weights that are no longer uniformly low with respect to the weights of the particles described in (b).
13. A mobile device tracking system as claimed in claim 12 wherein the erroneous particles are selected so as to avoid introduction of additional error into the current estimate of the mobile device's position.
14. A mobile device tracking system as claimed in claim 12 wherein the mobile device is a robot.
15. A mobile device tracking system as claimed in claim 12 wherein the determination of delocalization includes calculating and monitoring the averaged weight or the median weight of the erroneous particles over multiple iterations.
16. A system for identifying and marking dynamic areas of a map of a physical environment of a mobile device, the system comprising:
- a. a data acquisition system that generates data identifying the mobile device's physical environment; and
- b. processing apparatus, responsive to the data acquisition system, to map or model the mobile device's physical environment and the data acquisition system's location within the mobile device's physical environment in a cell-based grid in which cells are assigned probabilities indicating certainty about whether the physical space corresponding to a cell is occupied by an object or contains empty space;
- c. wherein the processing apparatus includes apparatus that: i. assigns and updates probabilities to each cell within the grid map from the data generated by the data acquisition system; ii. determines if changes in a cell's probability of occupancy indicate that a cell currently identified as empty has become occupied or a cell currently identified as occupied has become empty; and iii. marks such cells so as not to be updated with regard to their probability of containing an obstacle or not while said probability is changing.
17. A system as claimed in claim 16 wherein the marked cells also include a zone, of predetermined size, surrounding the marked cells.
18. A system claimed in claim 16 wherein the mobile device is a robot, the system further comprising a distance-measuring device, said distance-measuring device comprising light-emitting and light-receiving apparatus to emit light to measure distance based on reflected light, said data acquisition system being responsive to said distance-measuring device to generate said data identifying the mobile device's physical environment.
19. A system as claimed in claim 18 wherein the distance-measuring device comprises a laser rangefinder.
20. A method for controlling movement of a robot, the method comprising:
- a. generating data identifying a physical environment of the robot;
- b. mapping or modeling the robot's physical environment using said data;
- c. determining whether the robot has lost its preferred orientation for operation;
- d. if the robot has lost its preferred orientation, suspending said mapping while the robot has lost its preferred orientation, and resuming said mapping when the robot has resumed its preferred orientation; and
- e. controlling movement of the robot in accordance with (a) to (d).
21. A method as claimed in claim 20 wherein said generating comprises measuring distances from objects in the robot's physical environment using a laser rangefinder.
22. A method as claimed in claim 20 wherein said determining comprises measuring delocalization of said robot with an accelerometer.
23. A method for controlling movement of a robot, the method comprising:
- a. generating data identifying the physical environment of the robot and the robot's location within the robot's physical environment;
- b. suspending or modifying the use of data generated by the data acquisition system when a portion of the map above a threshold limit corresponds to a predetermined shift of one or more elements in the map; and
- c. controlling movement of the robot in accordance with (a) and (b).
24. A method for controlling movement of a robot, the method comprising:
- a. generating a map of the robot's physical environment, said generating occurring either concurrently with or prior to tracking the robot's position;
- b. updating said map either concurrently with or prior to tracking the device's position;
- c. determining the robot's current location within said map by generating particles, each of said particles representing a potential position and/or orientation of the robot within its physical environment, wherein a data set of said particles may be generated and maintained iteratively to track changing position of the device within its physical environment, said determining including:
- i. assigning a weight to each particle, particle weight being a relative measure of the particle's likelihood of accurately representing the robot's position with respect to other particles; and
- ii. introducing erroneous particles whose potential positions are selected so that their assigned weights should be uniformly low with respect to the weights of the particles representing a potential position and/or orientation of the robot; and
- iii. comparing the weights of the erroneous particles and the weights of the particles representing a potential position and/or orientation of the robot to determine whether the robot has become delocalized when a substantial number of erroneous particles have weights that are no longer uniformly low with respect to the weights of the particles representing a potential position and/or orientation of the robot.
- d. The method further comprising controlling movement of the robot in accordance with (a) to (c).
25. A method for controlling movement of a robot, the method comprising:
- a. generating data identifying the robot's physical environment; and
- b. mapping or modeling the robot's physical environment and location within the robot's physical environment in a cell-based grid in which cells are assigned probabilities indicating certainty about whether the physical space corresponding to a cell is occupied by an object or contains empty space;
- c. assigning and updating probabilities to each cell within the grid map from the generated data;
- d. determining whether changes in a cell's probability of occupancy indicate that a cell currently identified as empty has become occupied or a cell currently identified as occupied has become empty;
- e. marking such cells so as not to be updated with regard to their probability of containing an obstacle or not while said probability is changing; and
- f. controlling movement of the robot in accordance with (a) to (c).
Type: Application
Filed: Aug 31, 2010
Publication Date: Apr 7, 2011
Applicant: NEATO ROBOTICS, INC. (Mountain View, CA)
Inventors: Boris SOFMAN (Pittsburgh, PA), Vladimir ERMAKOV (Santa Clara, CA), Mark EMMERICH (San Jose, CA), Steven ALEXANDER (Fremont, CA), Nathaniel David MONSON (Mountain View, CA)
Application Number: 12/873,018
International Classification: G06F 19/00 (20110101); G06F 15/00 (20060101); G06G 7/48 (20060101);