Monday 30 December 2019

google earth engine - Error -Collection query aborted after accumulating over 5000 elements


I have created a code that suppose to take sentinel images and calculate the NDVI for a speficif polygon I have. I need it to run it for 2 years, but I have realized that for some reason the dataset is way too big and I get wrong numbers, for example, if I ask for the size of the dataset between 01/01/2018 to 03/01/2018 I get that the size of the dataset is 5099 images which doesn't make any sense.


this is the code I have up to now:


/**
* Function to mask clouds using the Sentinel-2 QA band
* @param {ee.Image} image Sentinel-2 image
* @return {ee.Image} cloud masked Sentinel-2 image
*/
function maskS2clouds(image) {
var qa = image.select('QA60');


// Bits 10 and 11 are clouds and cirrus, respectively.
var cloudBitMask = 1 << 10;
var cirrusBitMask = 1 << 11;

// Both flags should be set to zero, indicating clear conditions.
var mask = qa.bitwiseAnd(cloudBitMask).eq(0)
.and(qa.bitwiseAnd(cirrusBitMask).eq(0));

return image.updateMask(mask).divide(10000);

}

// Map the function over one year of data and take the median.
// Load Sentinel-2 TOA reflectance data.
var dataset = ee.ImageCollection('COPERNICUS/S2')
.filterDate('2018-01-01', '2018-01-03')
// Pre-filter to get less cloudy granules.
.filter(ee.Filter.lt('CLOUDY_PIXEL_PERCENTAGE', 20))
.select('B2','B3','B4','B8','QA60')
.map(maskS2clouds);


var rgbVis = {
min: 0.0,
max: 0.3,
bands: ['B4', 'B3', 'B2'],
};

var clippedCol=dataset.map(function(im){
return im.clip(geometry);
});


//test if clipping the image collection worked
Map.centerObject(geometry);
Map.addLayer(clippedCol.median(), rgbVis, 'RGB');

// Get the number of images.
var count = dataset.size();
print('Count: ',count);
print(clippedCol);//here I get the error messege "collection query aborted after accumulation over 5000 elements
print(dataset,'dataset');//the same error here



//function to calculate NDVI
var addNDVI = function(image) {
var ndvi = image.normalizedDifference(['B8', 'B4']).rename('NDVI');
return image.addBands(ndvi);
};

//NDVI to the ckipped image collection
var withNDVI = clippedCol.map(addNDVI);


// Test the addNDVI function on a single image.
var ndvi1 = withNDVI.select('NDVI').mean();


var colorizedVis = {
min: 0.0,
max: 1.0,
palette: [
'FFFFFF', 'CE7E45', 'DF923D', 'F1B555', 'FCD163', '99B718', '74A901',

'66A000', '529400', '3E8601', '207401', '056201', '004C00', '023B01',
'012E01', '011D01', '011301'
],
};


Map.addLayer(ndvi1,colorizedVis,'test');
Map.centerObject(geometry);



my goal is in the end to see the NDVI mean value for every image.



Answer



First to address your question, the Error occurs when you are trying to print more than 5000 elements in your console which it says. Since you have 5099 images in specified date range, the error arises. Since you don't want to do a global analysis as you are clipping images, you can actually use


.filterBounds(geometry)

on your image collection along with your other filters so that you only get images within your region which should be much lower. As for viewing mean NDVI, i can see your layer without any issue. however if you are not seeing the NDVI layer at all then that is probably because the images in that date range in your area has cloud cover more than 20 percent which is filtered out by


.filter(ee.Filter.lt('CLOUDY_PIXEL_PERCENTAGE', 20))

in your code. You could try increasing the value 20 to something higher but you might still get empty pixels because of the cloud masking. So it might be a good idea to see the images first without removing clounds and then play with dates so that you get a good number of non-cloudy pixels.


No comments:

Post a Comment

arcpy - Changing output name when exporting data driven pages to JPG?

Is there a way to save the output JPG, changing the output file name to the page name, instead of page number? I mean changing the script fo...