0zgur0/multi-stage-convstar-network

[RSE 2021] Crop mapping from image time series: deep learning with multi-scale label hierarchies

ms-convSTAR

Pytorch implementation for hierarchical time series classification with multi-stage convolutional RNN described in:

Crop mapping from image time series: deep learning with multi-scale label hierarchies. Turkoglu, Mehmet Ozgur and D'Aronco, Stefano and Perich, Gregor and Liebisch, Frank and Streit, Constantin and Schindler, Konrad and Wegner, Jan Dirk. Remote Sensing of Environment, 2021.

[Paper] - [Poster]

If you find our work useful in your research, please consider citing our paper:

@article{turkoglu2021msconvstar,
  title={Crop mapping from image time series: deep learning with multi-scale label hierarchies},
  author={Turkoglu, Mehmet Ozgur and D'Aronco, Stefano and Perich, Gregor and Liebisch, Frank and Streit, Constantin and Schindler, Konrad and Wegner, Jan Dirk},
  journal={Remote Sensing of Environment},
  volume={264},
  year={2021},
  publisher={Elsevier}
}

ZueriCrop Dataset

Download the dataset via https://polybox.ethz.ch/index.php/s/uXfdr2AcXE3QNB6

Getting Started

Train the model e.g., for fold:1 with

python3 train.py --data /path/to/data --fold 1

Test the trained model e.g., for fold:1 with

python3 test.py --data /path/to/data --fold 1 --snapshot /path/to/trained_model

multi-stage-convstar-network's Issues

The problem about ZueriCrop dataset

We are very interested in this dataset but have some confusion for this dataset.
In this paper, I learned that the input image sizes are H = W = 24, B = 4 and sequence length is T = 71. But the size of "data" in the dataset is (27977,142,24,24,9). I am very confused about the time dimension (Second dimension). Why is not72 but is 142.

We look forward to your reply.

Spectral Bands order

Hello. What is the order of the spectral bands in your data if I want to use all 9 bands? Could you please tell me each of the 9 channels (bands) you have in your data corresponds to which spectral bands? For instance, the fourth channel corresponds to NIR? SWIR?....

52 Classes instead of 48

Hello. When I generate the data using your data loader, the label is indexed from 0 to 51, which is different from what has been mentioned in your paper. I investigated your code and I realized that the labels 0, 47, 25, and 35 belong to "0_unkown", "Waters", "Non agriculture", and "Special cultures" respectively. These classes are not among those 48 classes in the paper. To my understanding, to obtain the 48 classes mentioned in the paper, I should remove the labels 0, 47, 25, and 35 (and their corresponding pixels) from the data. Am I correct? Could you please confirm that?