5.2 Dealing with cloud in MERIS L2 data

Atmospheric correction of MERIS (and other optical satellite data) starts with an algorithm that identifies pixels containing cloud. This processing step generally identifies cloudy pixels based on their brightness, and raises a CLOUD flag, which may be used during the L2 processing in order to mask clouds.

Whilst it is easy to identify pixels that are fully cloud covered, partial cloud cover represents more of a problem. Edges of clouds, scattered small clouds and low level fog may be harder to identify, and L2 data sometimes contain sub-pixel cloud that has not been identified by the cloud-flagging algorithm. This causes problems for algorithms that rely on spectral ratios to calculate chlorophyll concentrations. I this section you will look at this problem in MERIS L2 data from the Benguela upwelling, and investigate ways of overcoming the problem of sub-pixel cloud that has not been flagged by the cloud-flagging routine.

Activity:

1. Open the file MER_RR__2COLRA20040201_~.N1.
2. In the left frame, select Bands and open the text-file algal_1 data set by double-clicking on it in the right frame.
3. When the Redisplay dialogue appears, accept the default settings; do not set the null values.

As you can see, the image contains not just water pixels showing chlorophyll concentrations; there are also cloud pixels showing cloud top pressure, and land pixels with the top of atmosphere vegetation index (TOAVI). Along both edges are strips where no radiance samples were available to fill the MERIS product grid.

Using MERIS class flags

To produce a gridded image with valid chlorophyll concentrations the contributing scenes must contain only valid data. All other pixels must be masked. The class flags provided with the image data contain the information you need for this task.

1. In the left frame, select Flag Codings and open the text-file l2_flags that appears in the right frame.
   Note: You have to open the text file before the flags data set, because they have the same name, and if the flags 'image' is already open, Bilko will not open the text file.

2. Open the l2_flags data set (the last data set in the 'Bands' folder). Accept the Redisplay default settings, they are not important at this stage.

The l2_flags text file is a list of named flags, each followed by a number. The numbers will allow you to use the l2_flags data set to mask all pixels containing unwanted data. 'MERIS level 2 flags' gives you a more detailed description of these flags and what they mean, along with the binary code of each flag.

The flags data set contains all the flags related to the different geophysical data sets in the 'Bands' folder. Several pixels have been given more than one flag. This is the case for instance with pixels given the flag value 3655697. The key to the flags are in the binary version of this number.

Question 1.
The binary version of of 3655697 is given in the table below. Which are the flags given to this pixel? (To help you the bits have been numbered from 0 to 23.)

The flag codes may be used to make a bitmask to remove invalid data from the algal_1 image. For instance, only pixels classified as water are of interest in a gridded image of chlorophyll concentrations, and the WATER flag allows you to select these pixels.

Activity: Using a class flag to select valid data
Bilko applies masks using formula documents. Once you know how these work, you can write a formula to suit any flag and data set you want. The following steps take you through the process of applying a bitmask formula to mask invalid pixels in the algal_1 data from the Benguela.

1. Connect the algal_1 and l2_flags images into a stack, adding one blank image (figure (5K)).
2. Right click on the stack and select zoom, and in the zoom dialogue click preserve shape , which allows you to see the whole image at once.
3. Minimise the single images that were used to make up the stack (algal_1 and l2_flags), but don't close them.
4. Open the formula mer2water.frm .

The formula follows the standard Bilko pattern (see figure (5K)). for details).

Question 2.
The formula uses bitwise and ( & ) to compare the flag code and the values given to the pixels in the l2_flags data set. Below is a data-values taken from the l2_flags image. Compare this with the flag code for WATER, and if the condition is satisfied:

if ((l2_flags&WATER)==WATER)        

Note: In the table above WATER = 2097152, a number where only one position (bit 21) has the value 1. The pixel's FLAG VALUE = 4196353, which has 1 in three different bit positions, each corresponding to a particular flag. If you are still unsure how to tackle this, 'Bitwise operators and Envisat flags' explains.

Having seen how the formula works, you wil now apply it to create a new image containing only water pixels.

1. Use the selector, or tab through the stack until the blank image is on top.
2. Activate the formula by clicking on it.
3. Select Options! from the menu bar, and make sure the output image is set to the same numerical format as algal_1, and that the special null handling box is unchecked (figure).
4. Copy the formula ( [CTRL+C] ) and paste it onto the onto the set of images ([CTRL+V]).

Notice how the blank image is filled with data where the WATER flag holds true, while remaining pixels are white (255). The white pixels are those that have been classified as either land or cloud. Save the output as mer_20040201_a1w.dat before you continue.

Using confidence flags to refine selection

You'll probably also have noticed a number of dark pixels, most of which are near the edge of clouds, or associated with cloud in some other way. If you zoom in (double click on the image) you can read the value of some of these pixels on the status bar; the majority of these have been given the value 0 (black). These are pixels where the atmospheric correction failed, probably due to sub-pixel cloud; so the 0 here represents non-valid data, not water with no chlorophyll. There are also many scattered pixels that appear darker than their surroundings; their pattern follows the lines of the cloud bands, so we can assume that they also contain sub-pixel cloud. If these pixels were sufficiently anomalous to case problems for the aerosol correction, they would be accompanied by a confidence flag.

A MERIS Confidence flag relates to the quality of the geophysical data in the pixel that contains the flag. Unlike class flags, which are mutually exclusive, confidence and science flags may be used in combination with each other and with the class flags. Often the conditions that give rise to one flag are also likely to trigger others. During processing from Level 1 (top of atmosphere radiances) to Level 2 (geophysical data) confidence flags are raised when

• there is a failure of the algorithm used to derive a geophysical parameter
• the algorithm runs, but conditions indicate that the result may not be valid

You can use confidence flags to modify the bitmask formula used to select valid pixels. For example, if the aerosol correction triggered a confidence flag for pixels containing scattered cloud, this flag may be used to mask the dark pixels you see in this algal_1 image.

Question 3
a)	Consult the table of MERIS level 2 flags. Which of the confidence flags do you feel might be most suitable for this task?
b)	How would you write a condition (an if statement) to check pixels for the presence of flag PCD_13? Explain when the condition would be true, and what instructions should be carried out by the program in this case.
c)	How would you add this condition to the existing formula?

Copy and paste the new formula to the stack and watch what happens to the image. Once you are sure the formula has worked, save it as mer2water_a1a.frm, and save a copy of the masked image created by the formula as mer_20040201_a1a.dat.

1. Minimize all windows except the stack, which you should display with the new output image on top,
2. open the image you saved earlier (mer_20040201_a1w.dat),
3. and use Windows > Tile Vertical to display the two images side by side.

Question 4.