Some bugs about the extract_values_to_points function

[zwy1502]

Some bugs about the extract_values_to_points function

[zwy1502]

<style> </style>

NDVI_1	NDVI_2	NDVI_3	NDVI_4	NDVI_5	NDVI_6	NDVI_7	NDVI_8	NDVI_9	NDVI_10	NDVI_11	NDVI_12	class	system:index	label
0.015651135	0.606169939	0.749406159	0.747073472	0.60399425	0.120363973	0.021767115	0.663969815	0.703009725	0.680603921	0.435596526	0.275629222	0	0	hhmc
0.015651135	0.606169939	0.749406159	0.747073472	0.60399425	0.120363973	0.021767115	0.663969815	0.703009725	0.680603921	0.435596526	0.275629222	0	1	hhmc
0.015651135	0.606169939	0.749406159	0.747073472	0.60399425	0.120363973	0.021767115	0.663969815	0.703009725	0.680603921	0.435596526	0.275629222	0	2	hhmc
0.015651135	0.606169939	0.749406159	0.747073472	0.60399425	0.120363973	0.021767115	0.663969815	0.703009725	0.680603921	0.435596526	0.275629222	0	3	hhmc
0.015651135	0.606169939	0.749406159	0.747073472	0.60399425	0.120363973	0.021767115	0.663969815	0.703009725	0.680603921	0.435596526	0.275629222	0	4	hhmc
0.015651135	0.606169939	0.749406159	0.747073472	0.60399425	0.120363973	0.021767115	0.663969815	0.703009725	0.680603921	0.435596526	0.275629222	0	5	hhmc
0.015651135	0.606169939	0.749406159	0.747073472	0.60399425	0.120363973	0.021767115	0.663969815	0.703009725	0.680603921	0.435596526	0.275629222	0	6	hhmc
0.015651135	0.606169939	0.749406159	0.747073472	0.60399425	0.120363973	0.021767115	0.663969815	0.703009725	0.680603921	0.435596526	0.275629222	0	7	hhmc
0.015651135	0.606169939	0.749406159	0.747073472	0.60399425	0.120363973	0.021767115	0.663969815	0.703009725	0.680603921	0.435596526	0.275629222	0	8	hhmc
0.015651135	0.606169939	0.749406159	0.747073472	0.60399425	0.120363973	0.021767115	0.663969815	0.703009725	0.680603921	0.435596526	0.275629222	0	9	hhmc
0.015651135	0.606169939	0.749406159	0.747073472	0.60399425	0.120363973	0.021767115	0.663969815	0.703009725	0.680603921	0.435596526	0.275629222	0	0000000000000000000a	hhmc
0.015651135	0.606169939	0.749406159	0.747073472	0.60399425	0.120363973	0.021767115	0.663969815	0.703009725	0.680603921	0.435596526	0.275629222	0	0000000000000000000b	hhmc
0.015651135	0.606169939	0.749406159	0.747073472	0.60399425	0.120363973	0.021767115	0.663969815	0.703009725	0.680603921	0.435596526	0.275629222	0	0000000000000000000c	hhmc
0.015651135	0.606169939	0.749406159	0.747073472	0.60399425	0.120363973	0.021767115	0.663969815	0.703009725	0.680603921	0.435596526	0.275629222	0	0000000000000000000d	hhmc
0.015651135	0.606169939	0.749406159	0.747073472	0.60399425	0.120363973	0.021767115	0.663969815	0.703009725	0.680603921	0.435596526	0.275629222	0	0000000000000000000e	hhmc
0.015651135	0.606169939	0.749406159	0.747073472	0.60399425	0.120363973	0.021767115	0.663969815	0.703009725	0.680603921	0.435596526	0.275629222	0	0000000000000000000f	hhmc
0.015651135	0.606169939	0.749406159	0.747073472	0.60399425	0.120363973	0.021767115	0.663969815	0.703009725	0.680603921	0.435596526	0.275629222	0	10	hhmc
0.015651135	0.606169939	0.749406159	0.747073472	0.60399425	0.120363973	0.021767115	0.663969815	0.703009725	0.680603921	0.435596526	0.275629222	0	11	hhmc
0.015651135	0.606169939	0.749406159	0.747073472	0.60399425	0.120363973	0.021767115	0.663969815	0.703009725	0.680603921	0.435596526	0.275629222	0	12	hhmc
0.015651135	0.606169939	0.749406159	0.747073472	0.60399425	0.120363973	0.021767115	0.663969815	0.703009725	0.680603921	0.435596526	0.275629222	0	13	hhmc
0.015651135	0.606169939	0.749406159	0.747073472	0.60399425	0.120363973	0.021767115	0.663969815	0.703009725	0.680603921	0.435596526	0.275629222	0	14	hhmc
0.015651135	0.606169939	0.749406159	0.747073472	0.60399425	0.120363973	0.021767115	0.663969815	0.703009725	0.680603921	0.435596526	0.275629222	0	15	hhmc
0.015651135	0.606169939	0.749406159	0.747073472	0.60399425	0.120363973	0.021767115	0.663969815	0.703009725	0.680603921	0.435596526	0.275629222	0	16	hhmc
0.015651135	0.606169939	0.749406159	0.747073472	0.60399425	0.120363973	0.021767115	0.663969815	0.703009725	0.680603921	0.435596526	0.275629222	0	17	hhmc
0.015651135	0.606169939	0.749406159	0.747073472	0.60399425	0.120363973	0.021767115	0.663969815	0.703009725	0.680603921	0.435596526	0.275629222	0	18	hhmc
0.015651135	0.606169939	0.749406159	0.747073472	0.60399425	0.120363973	0.021767115	0.663969815	0.703009725	0.680603921	0.435596526	0.275629222	0	19	hhmc
0.015651135	0.606169939	0.749406159	0.747073472	0.60399425	0.120363973	0.021767115	0.663969815	0.703009725	0.680603921	0.435596526	0.275629222	0	0000000000000000001a	hhmc
0.015651135	0.606169939	0.749406159	0.747073472	0.60399425	0.120363973	0.021767115	0.663969815	0.703009725	0.680603921	0.435596526	0.275629222	0	0000000000000000001b	hhmc
0.015651135	0.606169939	0.749406159	0.747073472	0.60399425	0.120363973	0.021767115	0.663969815	0.703009725	0.680603921	0.435596526	0.275629222	0	0000000000000000001c	hhmc
0.015651135	0.606169939	0.749406159	0.747073472	0.60399425	0.120363973	0.021767115	0.663969815	0.703009725	0.680603921	0.435596526	0.275629222	0	0000000000000000001d	hhmc
0.015651135	0.606169939	0.749406159	0.747073472	0.60399425	0.120363973	0.021767115	0.663969815	0.703009725	0.680603921	0.435596526	0.275629222	0	0000000000000000001e	hhmc
0.015651135	0.606169939	0.749406159	0.747073472	0.60399425	0.120363973	0.021767115	0.663969815	0.703009725	0.680603921	0.435596526	0.275629222	0	0000000000000000001f	hhmc

The above is my output result. The first problem is that the pixel values extracted by this function are exactly the same. This must be a problem. I have verified that the image is normal. The second problem is that there will always be something similar to 0000000000000000000a.
0000000000000000000b,
0000000000000000000c,
0000000000000000000d,
0000000000000000000e,
0000000000000000000f and other abnormal indexes, why is this happening?

[giswqs]

Please provide a complete script that can reproduce the issue. Make sure you make the asset publicly accessible. And the variable studyArea is not defined.

[zwy1502]

#This is my complete script

import geemap
import ee
geemap.set_proxy(port='4780')
Map = geemap.Map()

Map.add_tile_layer(
    url="https://mt1.google.com/vt/lyrs=y&x={x}&y={y}&z={z}&scale=2",
    name="HD Google Basemap",
)
studyArea = ee.FeatureCollection("projects/ee-my-zwy/assets/sampleShp")
hhmc = ee.FeatureCollection("projects/ee-my-zwy/assets/yb2020hhmc")
def maskS2clouds(image):
    cloudProb = image.select('MSK_CLDPRB')
    snowProb = image.select('MSK_SNWPRB')
    cloud = cloudProb.lt(10) # 云覆盖量小于10%
    scl = image.select('SCL')
    shadow = scl.eq(3) # 3 = 云阴影
    cirrus = scl.eq(10) # 10 = 卷云
    # 云概率小于10%或云阴影分类
    mask = cloud.And(cirrus.Not()).And(shadow.Not())
    return image.updateMask(mask).divide(10000).select("B.*").set(image.toDictionary(image.propertyNames()))


startDate = ee.Date('2020-01-01')
endDate = ee.Date('2021-01-01')
s2 = ee.ImageCollection('COPERNICUS/S2_SR_HARMONIZED').filterDate(startDate, endDate).filterBounds(studyArea).map(maskS2clouds)


def addIndices(image):
    ndvi = image.normalizedDifference(['B8', 'B4']).rename('NDVI')
    lswi = image.normalizedDifference(['B8', 'B12']).rename('LSWI')
    rvi = image.select('B8').divide(image.select('B4')).rename('RVI')
    mndwi = image.normalizedDifference(['B3', 'B11']).rename('mNDWI')
    return image.addBands([ndvi, lswi, rvi, mndwi])
bandarray=['B2','B3','B4','B5','B6','B7', 'B8','B8A','B11','B12','NDVI','LSWI','RVI','mNDWI']

s2 = s2.map(addIndices).select(bandarray)
def monthly_median_composite(collection,year):
    composites = []
    for month in range(1, 13):

        startDate = ee.Date.fromYMD(year, month, 1)
        endDate = startDate.advance(1, 'month')

        month_s2 = collection.filterDate(startDate, endDate)

        medianComposite = month_s2.median().clipToCollection(studyArea)
        composites.append(medianComposite.set('month', month))
    return composites


monthly_composites_2020 = monthly_median_composite(s2,2020)
monthly_median_collection2020 = ee.ImageCollection.fromImages(monthly_composites_2020)

ndvi_collection = monthly_median_collection2020.select(['NDVI'])

ndvi_multi_band_image = ndvi_collection.toBands()


months = ['1', '2', '3', '4', '5', '6', '7', '8', '9', '10', '11', '12']


new_band_names = [f'NDVI_{month}' for month in months]


ndvi_multi_band_image = ndvi_multi_band_image.rename(new_band_names)

geemap.extract_values_to_points(hhmc, ndvi_multi_band_image, r'E:\download\ndvi_multi_band_imagehhmc.csv'

Now I find that I can correctly export pixel values using this method without using the geemap function. I am wondering why the extract_values_to_points function cannot do it. Here are the pixel values I extracted using other methods, and the result is correct. code show as below:

def addIndex(fc):
     # Use ee.List.sequence to create a sequence with the same size as FeatureCollection
     indices = ee.List.sequence(0, fc.size().subtract(1))
    
     # Create a new FeatureCollection that will contain the index corresponding to the original Feature
     indexed_fc = fc.toList(fc.size()).zip(indices).map(lambda el: ee.Feature(ee.List(el).get(0)).set('custom_index', ee.List( el).get(1)))
    
     #Convert the list back to FeatureCollection
     return ee.FeatureCollection(indexed_fc)

#Apply this function to the original FeatureCollection
hhmc= addIndex(hhmc)

geemap.ee_to_df(hhmc)
# Use the indexed sample collection to reduceRegions
def extract_and_export_samples(indexedSampleCollection, image, description):
     # Use the reduceRegions method to extract pixel values
     extracted = image.reduceRegions(collection=indexedSampleCollection, reducer=ee.Reducer.first(), scale=10)

     # At this stage, each feature in extracted should have a 'custom_index' attribute
     #Keep this 'custom_index' when exporting
     task = ee.batch.Export.table.toDrive(collection=extracted,
                                          description=description,
                                          selectors=['custom_index','class','label','NDVI_1', 'NDVI_2', 'NDVI_3', 'NDVI_4', 'NDVI_5', 'NDVI_6', 'NDVI_7', 'NDVI_8', 'NDVI_9' , 'NDVI_10', 'NDVI_11', 'NDVI_12'], #Add other required attribute names here
                                          fileFormat='CSV')
     task.start()

# Call the function and pass in the indexed sample collection
extract_and_export_samples(lw, ndvi_multi_band_image, 'LW_Sample_Values_2020')

[zwy1502]

In this way, I avoid using the extract_values_to_points function to get the correct result. At least the extracted pixel values will not be completely consistent, but this is not what I want because Professor Wu's function is simpler. Unfortunately I can't use it and I don't know what the problem is. Can you please teach me, Mr. Wu? My studyarea and sample hhmc were drawn manually on GEE and then uploaded to my own asset for use. There will be no problem of non-disclosure of assets.

I still have a small question. Is it now impossible to view the task progress on the GEE code editor when exporting GEE data? I also recently tried to export GEE data to Google Drive but I did not see the gear in the code editor. After a while, I saw the exported data in my Google Drive. How is this going?

[giswqs]

Make the assets publicly accessible:

• ee.FeatureCollection("projects/ee-my-zwy/assets/sampleShp")
• ee.FeatureCollection("projects/ee-my-zwy/assets/yb2020hhmc")

[giswqs]

Try setting the scale parameter for the extract_values_to_points function. It is likely that your workflow is computationally intensive. Earth Engine may use a coarser resolution for the computation.

[zwy1502]

Sorry, teacher, I didn't see the message before. Now I have tried to make these two assets public. Teacher Wu, can you help me take a look? I don't want to change the resolution because my requirement is 10m resolution. The links for these two assets are https://code.earthengine.google.com/?asset=projects/ee-my-zwy/assets/yb2020hhmc and https://code.earthengine.google.com/?asset= projects/ee-my-zwy/assets/sampleShp

[zwy1502]

Try setting the scale parameter for the extract_values_to_points function. It is likely that your workflow is computationally intensive. Earth Engine may use a coarser resolution for the computation.尝试为 extract_values_to_points 函数设置 scale 参数。您的工作流程可能属于计算密集型。 Earth Engine 可以使用较粗的分辨率进行计算。

Sorry, teacher, I didn't see the message before. Now I have tried to make these two assets public. Teacher Wu, can you help me take a look? I don't want to change the resolution because my requirement is 10m resolution. The links for these two assets are https://code.earthengine.google.com/?asset=projects/ee-my-zwy/assets/yb2020hhmc and https://code.earthengine.google.com/?asset= projects/ee-my-zwy/assets/sampleShp

[giswqs]

geemap.extract_values_to_points(hhmc, ndvi_multi_band_image, 'dvi_multi_band_imagehhmc.csv', scale=10)