dynaslum/satsense

Satsense is a Python library for land use/cover classification using satellite imagery

Satsense

Satsense is an open source Python library for patch based land-use and land-cover classification, initially developed for a project on deprived neighborhood detection. However, many of the algorithms made available through Satsense can be applied in other domains, such as ecology and climate science.

Satsense is based on readily available open source libraries, such as opencv for machine learning and the rasterio/gdal and netcdf libraries for data access. It has a modular design that makes it easy to add your own hand-crafted feature or use deep learning instead.

Detection of deprived neighborhoods is a land-use classification problem that is traditionally solved using hand crafted features like HoG, Lacunarity, NDXI, Pantex, Texton, and SIFT, computed from very high resolution satellite images. One of the goals of Satsense is to facilitate assessing the performance of these features on practical applications. To achieve this Satsense provides an easy to use open source reference implementation for these and other features, as well as facilities to distribute feature computation over multiple cpu’s. In the future the library will also provide easy access to metrics for assessing algorithm performance.

• satsense - library for analysing satellite images, performance evaluation, etc.
• notebooks - IPython notebooks for illustrating and testing the usage of Satsense

We are using python 3.6/3.7 and jupyter notebook for our code.

Documentation

Can be found on readthedocs.

Installation

Please see the installation guide on readthedocs.

Contributing

Contributions are very welcome! Please see CONTRIBUTING.md for our contribution guidelines.

Citing Satsense

If you use Satsense for scientific research, please cite it. You can download citation files from research-software.nl.

References

The collection of algorithms made available trough this package is inspired by

10. Graesser, A. Cheriyadat, R. R. Vatsavai, V. Chandola,

J. Long and E. Bright, "Image Based Characterization of Formal and Informal Neighborhoods in an Urban Landscape", in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 5, no. 4, pp. 1164-1176, Aug. 2012. doi: 10.1109/JSTARS.2012.2190383

Jordan Graesser himself also maintains a library with many of these algorithms.

Test Data

The test data has been extracted from the Copernicus Sentinel data 2018.

satsense's Issues

QGIS plugin

As a Remote sensing QGIS user
I would like to Use the feature extraction capabilities of satsense from QGIS
because I would like to use these features, but do not have experience in programming python.

Check if it is feasible to make it possible to use satsense as a QGIS plugin and do so if the resulting installation procedure is not too complicated.

Update scipy

The function scipy.misc.imread has been removed from scipy since version 1.3, but we use it in the function satsense.util.mask.load_mask_from_file.

Model validation using detected slums

As a agent based model researcher
I want to Validate my model using a complete set of slums
because I want to make sure my model is correct

Handle masked image pixels correctly

We need a strategy to handle masked image pixels values in the feature computations. This is probably different per feature if you want to do something meaningful with masked image data. The simplest and safest strategy is probably to return a masked or fill value instead of the feature value(s) if the input image contains any masked value.

Improve NetCDF conventions

We should add more netcdf conventions to the writing of files.

http://cfconventions.org/Data/cf-conventions/cf-conventions-1.7/build/ch05s06.html

There is a checker for nc files here:
http://cfconventions.org/compliance-checker.html

Clean up feature tests

Maybe it would be nice to use parametrized fixtures or use pytest.mark.parametrize to reduce the amount of duplicated code in the test functions.

Documentation/ipython notebook wrongly uses texton for pantex results

The feature extraction example uses texton to (partly) describe the pantex example output, both in the text as in the subcaptions of the figure.

https://satsense.readthedocs.io/en/latest/notebooks/feature_extraction.html#texton

Test Image class

Write unit tests for the satsense.image.Image class

Update satsense.util.mask module

Update the satsense.util.mask module so it works with the new design

Machine Learning Algorithm Training

As a Remote sensing expert
I want to Train a machine learning algorithm using the calculated features
because I can then apply the algorithm to detect slums.

Estimate: 3

Update Texton, Sift and features

As a Remote sensing expert
I want to Calculate the texton and sift features on a large satellite image on a computer with limited memory
because I do not have access to a computer with enough memory

Estimate: 5

Improve memory behaviour and simplify design

Work on this is done in #14

• Prevent copying a large numpy arrays between processes
• Use python slices instead of windows
• Check speed of rasterio windows against loading the image in bands (-> this was too slow to use for window generation)

Improve performance assessment functionality

One idea was to have a test based on area + location of the classified result.

NetCDF: HDF error

In case you are experiencing NetCDF: HDF errors after installation, see rasterio/rasterio#1574.

Implement BalancedGenerator

Make satsense citable

• Make a version 0.8 release as is
• Create a doi on zotero
• Add to https://www.research-software.nl

SIFT no longer available in opencv > 3.4.2

From travis:

cv2.error: OpenCV(3.4.3) /io/opencv_contrib/modules/xfeatures2d/src/sift.cpp:1207: error: (-213:The function/feature is not implemented) This algorithm is patented and is excluded in this configuration; Set OPENCV_ENABLE_NONFREE CMake option and rebuild the library in function 'create'

Add windows CI build

As a Satsense developer
I would like to add a windows build to the automated build system
so that I can be sure that satsense works correctly on windows systems

Windows on appveyor (Boatswain has appeveyor config as an example)

Have a look at skimage.feature.hog

We have our own implementation of the HoG feature, but there is also an implementation available in scikit-image:
https://scikit-image.org/docs/stable/api/skimage.feature.html#hog

Maybe we could use that instead.

Improve installation procedure

• Make it possible to install satsense with pip. Upload from travis?
• Create a package on conda-forge too?

Calculate features

As a remote sensing expert
I want to Calculate the features in satsense on a satellite image.
because This allows me to use them to detect slums

Estimate: 2

Add unit tests of feature computation

List of features that need tests:

• hog
• lacunarity
• ndxi
• pantex
• rid (do we believe this feature is worth the effort?)
• sift
• texton

The minimum test that should be available is a regression test. More is better, but maybe not feasible in the limited amount of time we have.

We may also consider comparing with the features as computed by spfeas.

Clean up notebooks

As a Remote sensing expert user of satsense
I would like to Have an example notebook showing how to use satsense
because I can use this as a starting point for my own research.

Most notebooks are outdated. Update or remove once we have a more stable API.

This notebook is most informative and relatively up to date:
satsense/notebooks/demo_notebooks/Classification/Simple Example.ipynb

User Documentation

As a remote sensing user of satsense
I would like to be able to look up how to use the relevant parts of satsense in the documentation
because without documentation using satsense will be very hard

• Check public interface for docstrings
• Example Jupyter Notebooks #17
• Write User Guide
• Readthedocs documentation (sphinx)

Estimate: 8 (timeboxed to 5)

Create automated pypi release

We should make sure the version is set correctly

Add geolocation information to FeatureVector saved as NetCDF

Partly done in the develop branch.

Use satsense for Sentinel 2 tif

As a user,
I want to run satsense over a few sentinel-2 tiles, and osm shapefile as my ground label
because I want to do landcover classification.

Apply ML algorithm

As a Remote sensing expert
I want to Apply the trained machine learning algorithm on a satellite image to detect slums
because I can then give this result as a validation set to the model builder

NEEDS REFINEMENT

Loading extracted features with masks, mask is not loaded

When loading computed features at a later time, the mask is not properly loaded...
This cost me quite some time to figure out what was going on with my features (all my statistics were messed up, +infinity for the means, etc)

I was loading features from .tif files.
I did look at the feature loading/saving code but it seemed like there is code for handling the mask...

This is the temporary fix I used (which assumes the mask fill_value = 1e+20):

x_test_mask = np.ma.masked_values(x_test, 1e+20)
mask = x_test_mask.mask

vector_mask = np.any(mask, axis=1)

index_vector = ~vector_mask
x_test = x_test[index_vector]
y_test = y_test[index_vector]

Improve FullGenerator unit tests

There is some suspicion that the generated windows are slightly shifted wrt to the edge of the image.

Update scikit-image

The skimage.feature.canny function was changed in version 0.15 of scikit-image. We need to update our unit tests and check that it still works as expected before we can update scikit-image.

Check that canny edge and texton descriptor images work with smaller chunks of the image

We need a careful look at whether or not the canny edge and texton descriptor images are acceptably similar when computed on smaller chunks of the image instead of on the entire image.

It would also be good to generalize this, so it is easy to add image types that require access to information about the entire image.

issue from install python lightgbm library in jupyter

please solve issue argent

Add mac build to travis

As a Satsense developer
I would like to add a mac build to the automated build system
so that I can be sure that satsense works correctly on mac systems

Improve save/load of FeatureVector

As a Remote sensing user
I would like to be able to restore a saved FeatureVector instance from file (netcdf or geotif) for Texton and Sift features.
because then I do not have to recalculate the initialization of these features to calculate them.

Explain Image.precompute_normalization() and automate in from_images methods

Improve FeatureSet

The satsense.features.FeatureSet class should provide a convenient way for users to combine satsense.image.FeatureVector or satsense.features.Feature objects into an array that can be used for machine learning tasks.

The set should allow;

• easy addition and removal of features or feature vectors
• automatic reshaping from a number of feature vectors of shape (width, height, window_number, feature_length) into (n_samples, combined_feature_size)
• support computing and loading from disk or mix of these operations
• train/test/validation split

Enable doctests?

As a user
I want to read examples in the documentation that work
because I will have trouble using satsense if they don't

We should probably enable doctests, this can be done by adding --doctest-modules under addopts in setup.cfg.