lkruitwagen/deepsentinel

DeepSentinel: a sentinel-1 and -2 self-supervised sensor fusion model for general purpose semantic embedding

Hello!

My name is Lucas, I use machine learning to build public information goods for sustainable finance. I'm the CTO of TransitionZero, an NGO accelerating the transition to a net-zero energy system. I was a postdoc with the University of Oxford School of Geography and the Environment Sustainable Finance Programme and Institute for New Economic Thinking. I was also the co-founder of Oxford Earth Observation, a environmental risk intelligence start-up. Please get in touch or see my bio page for more details!

I'm currently working on...

🔮 TransitionZero's Future Energy Outlook. Stay tuned...

Past work includes...

Oxford Earth Observation: Measuring and predicting water stress risk at the asset level.
💧 Wave2Web Hackathon: Predicting water availability for sixty-six reservoirs in India [Org][Dash][API][Paper]
🌾 FDL Europe 2021: Self-supervised embeddings of timeseries Sentinel-2 imagery for agriculture applications.[Presentation]
🎆 ML4CC: MLOps for climate change research, focussing on flood detection and projection
🤖 DeepSentinel: a general-purpose computer vision model for Sentinel-1 and -2 sensor fusion [Presentation][Paper][Repo]
☁️ FDL Europe 2020 Clouds & Aerosols: Machine learning for causal discovery of cloud & aerosol interactions [Tech Memo]
🌐 A complex network arrangement of the global fossil fuel supply chain [Blog Post][Paper][Repo]
☀️ A global inventory of solar PV generating units [Presentation] [GRASFI 2019] [NeurIPS 2019 CCAI][Nature][The Economist]

I also teach...

👀 Computer Vision: Masters' level introduction to computer vision with convolutional neural networks, transfer learning, and TensorFlow.
📐 Spatial Data Analysis with Python: Postgraduate-level spatial data analysis and spatial statistics.

About me

🇨🇦 in London 🇬🇧
🚀 Sci-fi reader (Le Guin, Butler, Simmons)
⛺ Happiest outdoors

deepsentinel's Issues

How to use custom data for training?

Hi, I have my own sentinel-1 and sentinel-2 image data, containing tiff files. I have been trying to understand the procedure for custom data preparation and then training. Kindly guide me on how can I use my custom sentinel-1 and sentinel-2 tif format data with this repo.

? on landcover.py : categorical transform?

Hi Lucas,

I'm back at it again. I'm trying to re-use your landcover.py for my hydrocarbon classes. But I keep getting an error here on line 188.

Y[ii,np.isin(arrs['LC'][OFFSET:self.patch_size+OFFSET, OFFSET:self.patch_size+OFFSET],vv)] = 1
IndexError: too many indices for array: array is 3-dimensional, but 4 were indexed

On line 186, Y is created as a 3d matrix (7x128x128) but then I'm not sure what line 188 is trying to do ...

Pretraining: channel stat .json file or pretrain path's?

Starting new issue, as I'm trying to train now! But I need some guidance.

Having troubles with the pretrain step. I left the default options from your example ML_CONFIG.yaml file. And got an error about the channel stats file:
No such file or directory: '/home/jupyter/deepsentinel/data/channel_stats/MIsland_v1_junk.json'​
Is this not generated?

So I tried removing pretraining step by keeping pretrain: blank in ML_CONFIG.yaml but still got an error
No such file or directory: '/home/jupyter/deepsentinel/experiments/sacred/4/pretrained_model.pth'​

What should I be doing to either use or not use a pretrained model?

File formats for data folder?

Hello!
I've been exploring your code, and am specifically looking for examples of downloading satellite images from GEE directly to a GCP storage bucket. I'm stuck at the data subdirectory: what kind of files is the code expecting here? Looks like vae.py reads them into a dataframe, but are they actually TIFF's from GEE? or something else?

Thank you!

Approach

• ML test-bench style
• streaming data? or fixed corpus? (transfer costs excessive on Azure - probably want a fixed corpus)
• two corpuses: [DL S2 DLSR, S1] and [L2A, EES1]
• pytorch
• pytorch parallel generator: https://stanford.edu/~shervine/blog/pytorch-how-to-generate-data-parallel
• run self-supervised experiments, with supervised finishing on a joint header
• "cross validate on the same distribution as test"

DEM

https://dwtkns.com/srtm30m/

VM attach data disk

https://docs.microsoft.com/en-us/azure/virtual-machines/linux/attach-disk-portal

cloud-free?

Could probably quite trivially set a threshold for S2 cloud coverage:

deepsentinel/deepsentinel/utils/point_generator.py

Line 127 in baebe2b

S1_df, S2_L1C_df, S2_L2A_df = self.load_catalog(start_date + timedelta(days=self.CONFIG['orbit_period']*orbit))

Just filtering the catalog before sampling points.

How can I get the CLC_100m_colmap.json?

When I run your code,I have a question,How can I get the CLC_100m_colmap.json?
Thank you!

overpy endpoint

mocnik-science/osm-python-tools#7