esa-philab / major-tom Goto Github PK
View Code? Open in Web Editor NEWExpandable Datasets for Earth Observation
Home Page: https://huggingface.co/Major-TOM
Expandable Datasets for Earth Observation
Home Page: https://huggingface.co/Major-TOM
Hi! Thanks a lot for releasing the dataset! I have a question about matching S1 and S2 data.
Based on the current processing, we estimate around 75% of the grid cells which have Sentinel-2 data will be successfully sampled with Sentinel-1 data.
I wonder how this number is calculated. Per my calculation, around 50% of the S2 grids can be matched with an S1 grid within an offset of four days. Here is how I did it
gdf_s2.sort_values("timestamp", inplace=True)
gdf_s1.sort_values("timestamp", inplace=True)
gdf_matched = pd.merge_asof(left=gdf_s2, right=gdf_s1, suffixes=("_s2", "_s1"), by="grid_cell", direction='nearest', on="timestamp", tolerance=pd.Timedelta(pd.offsets.Day(4)), allow_exact_matches=False).dropna(subset="parquet_url_s1", axis=0)
Any clarification would be much appreciated!
I downloaded some data and noticed that some S2 data is completely black, e.g., grid cell 207D_1378R
or 438U_1009R
. The S1 data looks fine.
I used the filter_download
function that is provided in this repo, I tested with and without by_row
. I also tested Image.open(BytesIO(table[col][0].as_py())).show()
with the same result.
The tif files do not include a FillValue
. I assume 0 is used for NaN values?
Is it possible that some data got corrupted during the download or upload to HF?
Not sure how I did this, but I was going between L2A-1C:
Traceback (most recent call last):
File "/home/user/.local/lib/python3.11/site-packages/reacton/ipyvue.py", line 23, in handler
callback_ref.current(*args)
File "/home/user/.local/lib/python3.11/site-packages/solara/components/button.py", line 84, in <lambda>
ipyvue.use_event(btn, click_event, lambda *_ignore: on_click and on_click())
^^^^^^^^^^
File "/app.py", line 54, in update_image
ret = map_to_image(m, return_centre=True, return_gridcell=True, l2a=(s2_level.value=='L2A'))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/helpers/functional.py", line 72, in map_to_image
bbox = map.get_bbox()
^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/leafmap/leafmap.py", line 4324, in get_bbox
bbox = [bounds[0][1], bounds[0][0], bounds[1][1], bounds[1][0]]
~~~~~~^^^
IndexError: tuple index out of range
Do you think you could provide a demo of filtering/query speed at different levels of image/request granularity?
Please include python code to recompute thumbnails and also how to compute full scale True Color Images from the three RGB bands, perhaps along the lines of https://sentinels.copernicus.eu/web/sentinel/user-guides/sentinel-2-msi/definitions. In practice, the details of the algorithm matter a great deal.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.