CLOUD DETECTION FROM SATELLITE IMAGERY

Abstract

Described herein are methods and systems for detecting clouds in satellite imagery captured using first and second sensor arrays that are carried by a satellite and physically offset from one another on the satellite. Movement mask data is produced based first image data and the second image data, obtained, respectively, using the first and second sensor arrays carried by the satellite. Cloud mask data is produced based on spectral information included in one of the first and second image data. Cloud detection data is produced based on the movement mask data and the cloud mask data, the cloud detection data indicating where it is likely, based on both the movement mask data and the cloud mask data, that one or more clouds are represented within one of the first and second image data. The cloud detection data can be used in various ways to account for the clouds included within the satellite imagery.

Description

Still referring to FIG. 6, step 606 involves producing cloud mask data based on spectral information included in one of the first and second image data. Such cloud mask data indicates where it is likely (based on the spectral information included in one of the first and second image data) that one or more clouds are represented within one of the first and second image data. For the sake of discussion, it is assumed that it is the first image data that is used to produce the cloud mask data at step 604. The spectral information included in the first image data can include, for example, B, G, R, and N1 band values for each pixel of N×M pixels included in the first image (e.g., 504 in FIG. 5A). As will be discussed in further detail below with reference to FIG. 8, in accordance with certain embodiments, step 606 can involve producing a cloud index value for each of a plurality of pixels (each of the N×M pixels included in the first image, or a subset thereof) based on R, G, and B band values (also referred to as RGB data) corresponding to those pixels, and thresholding the cloud index values.

As noted above, embodiments of the present technology take advantage of the fact that there is a physical offset between the first and second sensor arrays that are used to obtain the first and second image data respectively. More specifically, because the first sensor array used to obtain the first image data is physically offset on the satellite from the second sensor array used to obtain the second image data, a second satellite viewing angle associated with the second image data obtained using the second sensor array differs from a first satellite viewing angle associated with the first image data obtained using the first sensor array, with a difference between the first and second satellite viewing angles being a parallax angle. Further, because the first sensor array used to obtain the first image data is physically offset on the satellite from the second sensor array used to obtain the second image data, a second time (e.g., T2) at which part of the second image data corresponding to a piece (e.g., 518 in FIG. 5) of the geographic region (e.g., 520 in FIG. 5) is obtained using the second sensor array differs from a first time (e.g., T1) at which part of the first image data corresponding to the same piece (e.g., 518 in FIG. 5) of the geographic region (e.g., 520 in FIG. 5) is obtained using the first sensor array, with a difference between the first and second times being a temporal offset (i.e., temporal offset=T2−T1).

Certain embodiments of the present technology, described above, relate to a method for detecting clouds in satellite imagery captured using first and second sensor arrays that are carried by a satellite and physically offset from one another on the satellite. Such a method can include receiving first and second image data obtained, respectively, using the first and second sensor arrays carried by the satellite; producing movement mask data based on dissimilarities identified between the first image data and the second image data; producing cloud mask data based on spectral information included in one of the first and second image data; intersecting the movement mask data with the cloud mask data to thereby produce intersection data; and producing cloud detection data based on the intersection data.

Certain embodiments of the present technology are directed to a system for detecting clouds in satellite imagery captured using first and second sensor arrays that are carried by a satellite and physically offset from one another on the satellite, the system comprising one or more processors configured to: reduce movement mask data based first image data and the second image data, obtained, respectively, using the first and second sensor arrays carried by the satellite, the first image data including first image information about a first geographic region, and the second image data including second image information about the first geographic region, which differs from the first image information about the first geographic region included in the first image data. The one or more processors of the system or further configured to produce cloud mask data based on spectral information included in one of the first and second image data, the cloud mask data indicating where it is likely, based on the spectral information included in the one of the first and second image data, that one or more clouds are represented within the one of the first and second image data; and produce cloud detection data based on the movement mask data and the cloud mask data, the cloud detection data indicating where it is likely, based on both the movement mask data and the cloud mask data, that one or more clouds are represented within the one of the first and second image data. Additional details of such a system, which can include or be a computer system, can be appreciated from the above description.

Claims

1. A method for detecting clouds in satellite imagery captured using first and second sensor arrays that are carried by a satellite and physically offset from one another on the satellite, the method comprising:

receiving first and second image data obtained, respectively, using the first and second sensor arrays carried by the satellite;

producing movement mask data based on dissimilarities identified between the first image data and the second image data;

producing cloud mask data based on spectral information included in one of the first and second image data;

intersecting the movement mask data with the cloud mask data to thereby produce intersection data; and

producing cloud detection data based on the intersection data.

2. The method of claim 1, wherein:

the first image data includes first image information about a first geographic region;

the second image data includes second image information about the first geographic region, which differs from the first image information about the first geographic region included in the first image data;

the movement mask data indicates where it is likely that the dissimilarities between the first image data and the second image are due to movement of one or more objects within the first geographic region;

the cloud mask data indicates where it is likely, based on the spectral information included in the one of the first and second image data, that one or more clouds are represented within the one of the first and second image data; and

the cloud detection data indicates where it is likely, based on the dissimilarities between the first image data and the second image data, as well as based on the spectral information included in the one of the first and second image data, that one or more clouds are represented within the one of the first and second image data.

3. The method of claim 1, wherein because the first sensor array used to obtain the first image data is physically offset on the satellite from the second sensor array used to obtain the second image data:

a second satellite viewing angle associated with the second image data obtained using the second sensor array differs from a first satellite viewing angle associated with the first image data obtained using the first sensor array, with a difference between the first and second satellite viewing angles being a parallax angle; and

a second time at which part of the second image data corresponding to a piece of a first geographic region is obtained using the second sensor array differs from a first time at which part of the first image data corresponding to the piece of the first geographic region is obtained using the first sensor array, with a difference between the first and second times being a temporal offset.

4. The method of claim 1, wherein:

the first image data obtained using the first sensor array includes first spectral band data corresponding to a first spectral band,

the second image data obtained using the second sensor array includes second spectral band data corresponding to a second spectral band that is adjacent to and thereby proximate in wavelength to the first spectral band, and

the producing movement mask data comprises: performing edge detection on the first spectral band data and the second spectral band data to thereby produce first edge image data and second edge image data; computing dissimilarity values between corresponding image chips within the first and second edge image data; comparing the dissimilarity values to a threshold; and producing the movement mask data based on results of the comparing the dissimilarity values to the threshold.

5. The method of claim 1, wherein the producing cloud mask data based on spectral information included in one of the first and second image data, comprises:

computing chroma and lightness values for each pixel of a plurality of pixels of the one of the first and second image data;

computing a cloud index value, for each pixel of the plurality of pixels, based on the chroma and lightness values computed for the pixel;

comparing the computed cloud index value, for each pixel of the plurality of pixels, to a threshold value; and

classifying each pixel of the plurality of pixels as either corresponding to a cloud or not based on results of the comparisons to the threshold value.

6. The method of claim 5, wherein the computing the cloud index value, for each pixel of the plurality of pixels, is performing using the following equation: index = L a  C 2 + b

where index is the cloud index value for the pixel, L is the lightness value calculated for the pixel, C is the chroma value calculated for the pixel, and a and b are constants.

7. The method of claim 5, wherein the producing cloud mask data further comprises, after the classifying each pixel of the plurality of pixels as either corresponding to a cloud or not:

performing a binary opening morphological operation on results of the classifying, to thereby reduce a number of false positives included in the cloud mask data before the cloud mask data is intersected with the movement mask data to produce the intersection data.

8. The method of claim 1, wherein the producing cloud detection data, based on the intersection data, comprises:

identifying, based on the intersection data, one or more groups of pixels that may correspond to a cloud;

for each group of pixels that may correspond to a cloud, determining a magnitude of movement and a direction of movement of the group; and

clustering groups of pixels for which the determined magnitude of movement and the determined direction of movement are similar one another to thereby identify one or more distinct banks of clouds that are included within the first and second image data.

9. The method of claim 8, wherein, the producing the cloud detection data further comprises:

for each of the one or more distinct cloud banks identified as a result of the clustering, determining based on the first and second edge image data, additional pixels that correspond to the cloud bank;

classifying, as corresponding to a cloud, saturated pixels that are adjacent to another pixel that has already been classified as corresponding to a cloud;

performing a morphological fill operation to thereby classify, as corresponding to a cloud, pixels that are surrounded by other pixels that have already been classified as corresponding to a cloud;

up-sampling results of the morphological fill operation to increase resolution to an original resolution of the first and second image data;

performing a morphological dilation operation on results of the up-sampling to thereby produce dilated cloud detection data, wherein each pixel included in the dilated cloud detection data includes a respective cloud index value;

comparing the cloud index value, for each pixel of the dilated cloud detection data, to a threshold value; and

producing the cloud detection data based on results of the comparing the cloud index value, for each pixel of the dilated cloud detection data, to the threshold value.

10. A non-transitory processor readable storage device having processor readable code embodied on the processor read storage device, the processor readable code for programming one or more processors to perform a method comprising:

receiving first and second image data obtained, respectively, using the first and second sensor arrays carried by the satellite;

producing movement mask data based on dissimilarities identified between the first image data and the second image data;

producing cloud mask data based on spectral information included in one of the first and second image data;

intersecting the movement mask data with the cloud mask data to thereby produce intersection data; and

producing cloud detection data based on the intersection data.

11. The non-transitory processor readable storage device of claim 10,

wherein:

the first image data includes first image information about a first geographic region;

the second image data includes second image information about the first geographic region, which differs from the first image information about the first geographic region included in the first image data;

the movement mask data indicates where it is likely that the dissimilarities between the first image data and the second image are due to movement of one or more objects within the first geographic region;

the cloud mask data indicates where it is likely, based on the spectral information included in the one of the first and second image data, that one or more clouds are represented within the one of the first and second image data;

the cloud detection data indicates where it is likely, based on the dissimilarities between the first image data and the second image data, as well as based on the spectral information included in the one of the first and second image data, that one or more clouds are represented within the one of the first and second image data;

a second satellite viewing angle associated with the second image data obtained using the second sensor array differs from a first satellite viewing angle associated with the first image data obtained using the first sensor array, with a difference between the first and second satellite viewing angles being a parallax angle; and

a second time at which part of the second image data corresponding to a piece of the first geographic region is obtained using the second sensor array differs from a first time at which part of the first image data corresponding to the piece of the first geographic region is obtained using the first sensor array, with a difference between the first and second times being a temporal offset.

12. The non-transitory processor readable storage device of claim 10,

wherein:

the first image data obtained using the first sensor array includes first spectral band data corresponding to a first spectral band,

the second image data obtained using the second sensor array includes second spectral band data corresponding to a second spectral band that is adjacent to and thereby proximate in wavelength to the first spectral band, and

the producing movement mask data comprises: performing edge detection on the first spectral band data and the second spectral band data to thereby produce first edge image data and second edge image data; computing dissimilarity values between corresponding image chips within the first and second edge image data; comparing the dissimilarity values to a threshold; and producing the movement mask data based on results of the comparing the dissimilarity values to the threshold.

13. The non-transitory processor readable storage device of claim 10, wherein the producing cloud mask data based on spectral information included in one of the first and second image data, comprises:

computing chroma and lightness values for each pixel of a plurality of pixels of the one of the first and second image data;

computing a cloud index value, for each pixel of the plurality of pixels, based on the chroma and lightness values computed for the pixel;

comparing the computed cloud index value, for each pixel of the plurality of pixels, to a threshold value; and

classifying each pixel of the plurality of pixels as either corresponding to a cloud or not based on results of the comparisons to the threshold value.

14. The non-transitory processor readable storage device of claim 13, wherein the computing the cloud index value, for each pixel of the plurality of pixels, is performing using the following equation: index = L a  C 2 + b

where index is the cloud index value for the pixel, L is the lightness value calculated for the pixel, C is the chroma value calculated for the pixel, and a and b are constants.

15. The non-transitory processor readable storage device of claim 13, wherein the producing cloud mask data further comprises, after the classifying each pixel of the plurality of pixels as either corresponding to a cloud or not:

performing a binary opening morphological operation on results of the classifying, to thereby reduce a number of false positives included in the cloud mask data before the cloud mask data is intersected with the movement mask data to produce the intersection data.

16. The non-transitory processor readable storage device of claim 10, wherein the producing cloud detection data, based on the intersection data, comprises:

identifying, based on the intersection data, one or more groups of pixels that may correspond to a cloud;

for each group of pixels that may correspond to a cloud, determining a magnitude of movement and a direction of movement of the group; and

clustering groups of pixels for which the determined magnitude of movement and the determined direction of movement are similar one another to thereby identify one or more distinct banks of clouds that are included within the first and second image data.

17. The non-transitory processor readable storage device of claim 16, wherein the producing the cloud detection data further comprises:

for each of the one or more distinct cloud banks identified as a result of the clustering, determining based on the first and second edge image data, additional pixels that correspond to the cloud bank;

classifying, as corresponding to a cloud, saturated pixels that are adjacent to another pixel that has already been classified as corresponding to a cloud;

performing a morphological fill operation to thereby classify, as corresponding to a cloud, pixels that are surrounded by other pixels that have already been classified as corresponding to a cloud;

up-sampling results of the morphological fill operation to increase resolution to an original resolution of the first and second image data;

performing a morphological dilation operation on results of the up-sampling to thereby produce dilated cloud detection data, wherein each pixel included in the dilated cloud detection data includes a respective cloud index value;

comparing the cloud index value, for each pixel of the dilated cloud detection data, to a threshold value; and

producing the cloud detection data based on results of the comparing the cloud index value, for each pixel of the dilated cloud detection data, to the threshold value.

18. A method for detecting clouds in satellite imagery captured using first and second sensor arrays that are carried by a satellite and physically offset from one another on the satellite, the method comprising:

producing movement mask data based on first image data and the second image data, obtained, respectively, using the first and second sensor arrays carried by the satellite, the first image data including first image information about a first geographic region, and the second image data including second image information about the first geographic region, which differs from the first image information about the first geographic region included in the first image data;

producing cloud mask data based on spectral information included in one of the first and second image data, the cloud mask data indicating where it is likely, based on the spectral information included in the one of the first and second image data, that one or more clouds are represented within the one of the first and second image data; and

producing cloud detection data based on the movement mask data and the cloud mask data, the cloud detection data indicating where it is likely, based on both the movement mask data and the cloud mask data, that one or more clouds are represented within the one of the first and second image data.

19. The method of claim 18, wherein the producing movement mask data comprises:

performing edge detection on the first image data and the second image data to thereby produce first edge image data and second edge image data;

computing dissimilarity values between corresponding image chips within the first and second edge image data; and

producing the movement mask data based on results of comparing the dissimilarity values to a threshold.

20. The method of claim 18, wherein the producing cloud mask data based on spectral information included in one of the first and second image data, comprises:

computing color transformation values for each pixel of a plurality of pixels of the one of the first and second image data;

computing a cloud index value, for each pixel of the plurality of pixels, based on the color transformation values computed for the pixel;

comparing the computed cloud index value, for each pixel of the plurality of pixels, to a threshold value; and

classifying each pixel of the plurality of pixels as either corresponding to a cloud or not based on results of the comparisons to the threshold value.

21. A system for detecting clouds in satellite imagery captured using first and second sensor arrays that are carried by a satellite and physically offset from one another on the satellite, the system comprising:

one or more processors configured to produce movement mask data based on first image data and second image data, obtained, respectively, using the first and second sensor arrays carried by the satellite, the first image data including first image information about a first geographic region, and the second image data including second image information about the first geographic region, which differs from the first image information about the first geographic region included in the first image data;

produce cloud mask data based on spectral information included in one of the first and second image data, the cloud mask data indicating where it is likely, based on the spectral information included in the one of the first and second image data, that one or more clouds are represented within the one of the first and second image data; and

produce cloud detection data based on the movement mask data and the cloud mask data, the cloud detection data indicating where it is likely, based on both the movement mask data and the cloud mask data, that one or more clouds are represented within the one of the first and second image data.