remotesensinginfo/rsgislib

Remote Sensing and GIS Software Library; python module tools for processing spatial data.

rsgislib's Issues

rsgislib.imageutils.stretch_img_with_stats

Bug on MacOS - freezes for some reason which needs debugging.

Problem installing RSGISLIB

Recently I tried to install RSGISLib via Windows 10 through Visual Code Studio, this version:
Version: 1.81.1 (user setup)
Commit: 6c3e3dba23e8fadc360aed75ce363ba185c49794
Date: 2023-08-09T22:22:42.175Z
Electron: 22.3.18
ElectronBuildId: 22689846
Chromium: 108.0.5359.215
Node.js: 16.17.1
V8: 10.8.168.25-electron.0
OS: Windows_NT x64 10.0.19045

I followed your guide by typing:

conda create -n osgeo-env-v1 python=3.7
conda activate osgeo-env-v1 #changed from source activate osgeo-env-v1
conda install -c conda-forge rsgislib

But the installation got stuck here:

Any way to solve this?
Thank you very much

Confusion matrix for segmentation and classification example

I would like to know if there is a possibility to add the confusion matrix in the segmentation code example (03_intro_image_classification/02_segment_ml_classification.ipynb) with the 'rsgislib.classification.classaccuracymetrics.calc_acc_metrics_vecsamples' method. How would the implementation be to show the confusion matrix?

Error messages when running popImageStates on conda Windows 10

Hi all,

These error messages pop up when I run the command line below. The task still completes successfully, so it is probably a debugging leftover. Sorry for the encoding issues with the progress bar.

> rsgislib.imageutils.popImageStats(outfile, True, 0, True)

'test' is not recognized as an internal or external command, operable program or batch file.
'test' is not recognized as an internal or external command, operable program or batch file.
 ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓêΓûêΓûêΓûêΓûêΓûêΓûêΓûÅ 100.0% [44528/44528 | 1.4 kHz | 33s<0s] 

'test' is not recognized as an internal or external command, operable program or batch file.
'test' is not recognized as an internal or external command, operable program or batch file.
 ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓêΓûêΓûêΓûêΓûêΓûêΓûêΓûÅ 100.0% [44528/44528 | 869.6 Hz | 51s<0s]

Calculating Image Pyramids.

'test' is not recognized as an internal or external command, operable program or batch file.
'test' is not recognized as an internal or external command, operable program or batch file.
 ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûÅ 100.0% [ 100/ 100 | 8.4 Hz | 12s<0s]
 ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûÅ 100.0% [ 100/ 100 | 8.4 Hz | 12s<0s]

test_tools_ftptools failed with ftplib.error_proto

Hi,
When I tried to test the package with pytest, the test failed when calling the ftptools.upload_ftp_file function. Below is the error:

rsgislib/tools/ftptools.py:466: in upload_ftp_file
    ftp_conn.login(user=ftp_user, passwd=ftp_pass)
/usr/lib/python3.10/ftplib.py:414: in login
    resp = self.sendcmd('PASS ' + passwd)
/usr/lib/python3.10/ftplib.py:281: in sendcmd
    return self.getresp()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <ftplib.FTP object at 0x7f4f4c56dab0>

    def getresp(self):
        resp = self.getmultiline()
        if self.debugging:
            print('*resp*', self.sanitize(resp))
        self.lastresp = resp[:3]
        c = resp[:1]
        if c in {'1', '2', '3'}:
            return resp
        if c == '4':
            raise error_temp(resp)
        if c == '5':
            raise error_perm(resp)
>       raise error_proto(resp)
E       ftplib.error_proto: /usr/sbin/vsftpd: Relink `/lib64/libnss_myhostname.so.2' with `/lib64/librt.so.1' for IFUNC symbol `clock_gettime'

/usr/lib/python3.10/ftplib.py:255: error_proto

Below is the full log of pytest:

pytest log

==================================================================================================================================== test session starts ====================================================================================================================================
platform linux -- Python 3.10.4, pytest-7.1.1, pluggy-1.0.0
rootdir: /home/o0331dobe/Documents/AUR_packages/python-rsgislib/src/rsgislib-5.0.7/build-test
plugins: cov-2.12.1
collected 884 items                                                                                                                                                                                                                                                                         

tests/test_changedetect_pxloutlierchng.py s...                                                                                                                                                                                                                                        [  0%]
tests/test_classification.py ................                                                                                                                                                                                                                                         [  2%]
tests/test_classification_classaccuracymetrics.py ........                                                                                                                                                                                                                            [  3%]
tests/test_classification_classimblearn.py sssssssssssssss                                                                                                                                                                                                                            [  4%]
tests/test_classification_classlightgbm.py ssssss                                                                                                                                                                                                                                     [  5%]
tests/test_classification_classratutils.py ..                                                                                                                                                                                                                                         [  5%]
tests/test_classification_classsklearn.py ....                                                                                                                                                                                                                                        [  6%]
tests/test_classification_classxgboost.py sssssssss                                                                                                                                                                                                                                   [  7%]
tests/test_classification_clustersklearn.py ...                                                                                                                                                                                                                                       [  7%]
tests/test_elevation.py .............                                                                                                                                                                                                                                                 [  9%]
tests/test_image_morphology.py .............                                                                                                                                                                                                                                          [ 10%]
tests/test_imagecalc.py ..........................................................                                                                                                                                                                                                    [ 17%]
tests/test_imagecalc_calcindices.py .............                                                                                                                                                                                                                                     [ 18%]
tests/test_imagecalc_leastcostpath.py s                                                                                                                                                                                                                                               [ 18%]
tests/test_imagecalc_specunmixing.py ............ssssssss.....                                                                                                                                                                                                                        [ 21%]
tests/test_imagefilter.py .............................................................                                                                                                                                                                                               [ 28%]
tests/test_imageregistration.py ......                                                                                                                                                                                                                                                [ 29%]
tests/test_imageutils.py .....................................................................................................................................                                                                                                                        [ 44%]
tests/test_imageutils_imagelut.py ........                                                                                                                                                                                                                                            [ 45%]
tests/test_imageutils_stdimgblockiter.py ..                                                                                                                                                                                                                                           [ 45%]
tests/test_rastergis.py ........ss.................                                                                                                                                                                                                                                   [ 48%]
tests/test_regression.py .                                                                                                                                                                                                                                                            [ 48%]
tests/test_regression_regresssklearn.py .....                                                                                                                                                                                                                                         [ 48%]
tests/test_rsgislib_cli.py ........ssss.....................................s....                                                                                                                                                                                                     [ 55%]
tests/test_segentatation.py ...                                                                                                                                                                                                                                                       [ 55%]
tests/test_segentatation_shepherdseg.py ....                                                                                                                                                                                                                                          [ 55%]
tests/test_segentatation_skimgseg.py ssss                                                                                                                                                                                                                                             [ 56%]
tests/test_segentatation_tiledclump.py ....                                                                                                                                                                                                                                           [ 56%]
tests/test_timeseries_modelfitting.py ..                                                                                                                                                                                                                                              [ 57%]
tests/test_tools_checkdatasets.py .....................                                                                                                                                                                                                                               [ 59%]
tests/test_tools_filetools.py ..................................ss........                                                                                                                                                                                                            [ 64%]
tests/test_tools_ftptools.py FssFFss                                                                                                                                                                                                                                                  [ 65%]
tests/test_tools_geometrytools.py .......                                                                                                                                                                                                                                             [ 65%]
tests/test_tools_plotting.py ..............                                                                                                                                                                                                                                           [ 67%]
tests/test_tools_projection.py .........                                                                                                                                                                                                                                              [ 68%]
tests/test_tools_utils.py ..............................................................                                                                                                                                                                                              [ 75%]
tests/test_tools_visualisation.py s......ss.sss..ssssssss.                                                                                                                                                                                                                            [ 78%]
tests/test_vectorattrs.py .........................                                                                                                                                                                                                                                   [ 81%]
tests/test_vectorgeoms.py ..................s.......................                                                                                                                                                                                                                  [ 85%]
tests/test_vectorutils.py .....................................................................                                                                                                                                                                                       [ 93%]
tests/test_vectorutils_createrasters.py ......                                                                                                                                                                                                                                        [ 94%]
tests/test_vectorutils_createvectors.py .............                                                                                                                                                                                                                                 [ 95%]
tests/test_zonalstats.py .....................................                                                                                                                                                                                                                        [100%]

========================================================================================================================================= FAILURES ==========================================================================================================================================
___________________________________________________________________________________________________________________________________ test_upload_ftp_file ____________________________________________________________________________________________________________________________________

    def test_upload_ftp_file():
        import rsgislib.tools.ftptools
    
        test_file = os.path.join(TOOLS_UTILS_DATA_DIR, "basic_str.txt")
>       rsgislib.tools.ftptools.upload_ftp_file(
            test_file,
            ftp_url=ftp_url_ref,
            remote_path="/rsgislib_test/test",
            ftp_timeout=300,
            ftp_user=ftp_user_ref,
            ftp_pass=ftp_pass_ref,
            print_info=True,
        )

tests/test_tools_ftptools.py:30: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
rsgislib/tools/ftptools.py:466: in upload_ftp_file
    ftp_conn.login(user=ftp_user, passwd=ftp_pass)
/usr/lib/python3.10/ftplib.py:414: in login
    resp = self.sendcmd('PASS ' + passwd)
/usr/lib/python3.10/ftplib.py:281: in sendcmd
    return self.getresp()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <ftplib.FTP object at 0x7f4f5921c8b0>

    def getresp(self):
        resp = self.getmultiline()
        if self.debugging:
            print('*resp*', self.sanitize(resp))
        self.lastresp = resp[:3]
        c = resp[:1]
        if c in {'1', '2', '3'}:
            return resp
        if c == '4':
            raise error_temp(resp)
        if c == '5':
            raise error_perm(resp)
>       raise error_proto(resp)
E       ftplib.error_proto: /usr/sbin/vsftpd: Relink `/lib64/libnss_myhostname.so.2' with `/lib64/librt.so.1' for IFUNC symbol `clock_gettime'

/usr/lib/python3.10/ftplib.py:255: error_proto
----------------------------------------------------------------------------------------------------------------------------------- Captured stdout call ------------------------------------------------------------------------------------------------------------------------------------
Opening FTP Connection to ftp.dlptest.com
__________________________________________________________________________________________________________________________________ test_download_ftp_file ___________________________________________________________________________________________________________________________________

tmp_path = PosixPath('/tmp/pytest-of-o0331dobe/pytest-2/test_download_ftp_file0')

    def test_download_ftp_file(tmp_path):
        import rsgislib.tools.ftptools
    
        test_file = os.path.join(TOOLS_UTILS_DATA_DIR, "basic_str.txt")
>       rsgislib.tools.ftptools.upload_ftp_file(
            test_file,
            ftp_url=ftp_url_ref,
            remote_path="/rsgislib_test/test",
            ftp_timeout=300,
            ftp_user=ftp_user_ref,
            ftp_pass=ftp_pass_ref,
            print_info=True,
        )

tests/test_tools_ftptools.py:101: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
rsgislib/tools/ftptools.py:466: in upload_ftp_file
    ftp_conn.login(user=ftp_user, passwd=ftp_pass)
/usr/lib/python3.10/ftplib.py:414: in login
    resp = self.sendcmd('PASS ' + passwd)
/usr/lib/python3.10/ftplib.py:281: in sendcmd
    return self.getresp()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <ftplib.FTP object at 0x7f4f4c56dab0>

    def getresp(self):
        resp = self.getmultiline()
        if self.debugging:
            print('*resp*', self.sanitize(resp))
        self.lastresp = resp[:3]
        c = resp[:1]
        if c in {'1', '2', '3'}:
            return resp
        if c == '4':
            raise error_temp(resp)
        if c == '5':
            raise error_perm(resp)
>       raise error_proto(resp)
E       ftplib.error_proto: /usr/sbin/vsftpd: Relink `/lib64/libnss_myhostname.so.2' with `/lib64/librt.so.1' for IFUNC symbol `clock_gettime'

/usr/lib/python3.10/ftplib.py:255: error_proto
----------------------------------------------------------------------------------------------------------------------------------- Captured stdout call ------------------------------------------------------------------------------------------------------------------------------------
Opening FTP Connection to ftp.dlptest.com
________________________________________________________________________________________________________________________________ test_download_curl_ftp_file ________________________________________________________________________________________________________________________________

tmp_path = PosixPath('/tmp/pytest-of-o0331dobe/pytest-2/test_download_curl_ftp_file0')

    @pytest.mark.skipif(PYCURL_DB_NOT_AVAIL, reason="pycurl dependency not available")
    def test_download_curl_ftp_file(tmp_path):
        import rsgislib.tools.ftptools
    
        test_file = os.path.join(TOOLS_UTILS_DATA_DIR, "basic_str.txt")
>       rsgislib.tools.ftptools.upload_ftp_file(
            test_file,
            ftp_url=ftp_url_ref,
            remote_path="/rsgislib_test/test",
            ftp_timeout=300,
            ftp_user=ftp_user_ref,
            ftp_pass=ftp_pass_ref,
            print_info=True,
        )

tests/test_tools_ftptools.py:129: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
rsgislib/tools/ftptools.py:466: in upload_ftp_file
    ftp_conn.login(user=ftp_user, passwd=ftp_pass)
/usr/lib/python3.10/ftplib.py:414: in login
    resp = self.sendcmd('PASS ' + passwd)
/usr/lib/python3.10/ftplib.py:281: in sendcmd
    return self.getresp()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <ftplib.FTP object at 0x7f4f4c5068c0>

    def getresp(self):
        resp = self.getmultiline()
        if self.debugging:
            print('*resp*', self.sanitize(resp))
        self.lastresp = resp[:3]
        c = resp[:1]
        if c in {'1', '2', '3'}:
            return resp
        if c == '4':
            raise error_temp(resp)
        if c == '5':
            raise error_perm(resp)
>       raise error_proto(resp)
E       ftplib.error_proto: /usr/sbin/vsftpd: Relink `/lib64/libnss_myhostname.so.2' with `/lib64/librt.so.1' for IFUNC symbol `clock_gettime'

/usr/lib/python3.10/ftplib.py:255: error_proto
----------------------------------------------------------------------------------------------------------------------------------- Captured stdout call ------------------------------------------------------------------------------------------------------------------------------------
Opening FTP Connection to ftp.dlptest.com
===================================================================================================================================== warnings summary ======================================================================================================================================
../../../../../../../../usr/lib/python3.10/site-packages/geopandas/_compat.py:2
  /usr/lib/python3.10/site-packages/geopandas/_compat.py:2: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
    from distutils.version import LooseVersion

tests/test_changedetect_pxloutlierchng.py::test_find_class_kurt_skew_outliers
  /usr/lib/python3.10/site-packages/rios/imageio.py:63: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
    (numpy.bool,gdalconst.GDT_Byte),

tests/test_changedetect_pxloutlierchng.py::test_find_class_kurt_skew_outliers
  /usr/lib/python3.10/site-packages/rios/imageio.py:69: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
    (numpy.float,gdalconst.GDT_Float64)

tests/test_classification.py::test_get_class_training_data
  /home/o0331dobe/Documents/AUR_packages/python-rsgislib/src/rsgislib-5.0.7/build-test/rsgislib/vectorutils/__init__.py:252: DeprecationWarning: invalid escape sequence '\*'
    """

tests/test_classification.py::test_get_class_training_data
  /home/o0331dobe/Documents/AUR_packages/python-rsgislib/src/rsgislib-5.0.7/build-test/rsgislib/vectorutils/__init__.py:334: DeprecationWarning: invalid escape sequence '\*'
    """

tests/test_classification.py::test_get_class_training_data
  /home/o0331dobe/Documents/AUR_packages/python-rsgislib/src/rsgislib-5.0.7/build-test/rsgislib/vectorutils/__init__.py:428: DeprecationWarning: invalid escape sequence '\*'
    """

tests/test_classification.py: 10 warnings
tests/test_classification_classaccuracymetrics.py: 4 warnings
tests/test_vectorattrs.py: 9 warnings
tests/test_vectorgeoms.py: 9 warnings
tests/test_vectorutils.py: 59 warnings
tests/test_vectorutils_createrasters.py: 1 warning
tests/test_vectorutils_createvectors.py: 1 warning
  /usr/lib/python3.10/site-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
    pd.Int64Index,

tests/test_classification_classaccuracymetrics.py: 10 warnings
tests/test_vectorutils.py: 3 warnings
  /home/o0331dobe/Documents/AUR_packages/python-rsgislib/src/rsgislib-5.0.7/build-test/rsgislib/vectorutils/__init__.py:2765: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
    data_gdf = data_gdf.append(tmp_data_gdf)

tests/test_regression_regresssklearn.py::test_perform_search_param_opt
  /usr/lib/python3.10/site-packages/sklearn/model_selection/_search.py:292: UserWarning: The total space of parameters 1 is smaller than n_iter=25. Running 1 iterations. For exhaustive searches, use GridSearchCV.
    warnings.warn(

tests/test_regression_regresssklearn.py::test_perform_search_param_opt
tests/test_regression_regresssklearn.py::test_perform_search_param_opt
  /usr/lib/python3.10/site-packages/sklearn/model_selection/_validation.py:680: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples,), for example using ravel().
    estimator.fit(X_train, y_train, **fit_params)

tests/test_regression_regresssklearn.py::test_perform_search_param_opt
  /usr/lib/python3.10/site-packages/sklearn/model_selection/_search.py:926: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples,), for example using ravel().
    self.best_estimator_.fit(X, y, **fit_params)

tests/test_regression_regresssklearn.py::test_perform_kfold_fit
tests/test_regression_regresssklearn.py::test_perform_kfold_fit
tests/test_regression_regresssklearn.py::test_perform_kfold_fit
tests/test_regression_regresssklearn.py::test_perform_kfold_fit
  /home/o0331dobe/Documents/AUR_packages/python-rsgislib/src/rsgislib-5.0.7/build-test/rsgislib/regression/regresssklearn.py:273: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples,), for example using ravel().
    skl_regrs_obj.fit(x_train, y_train)

tests/test_regression_regresssklearn.py::test_apply_regress_sklearn_mdl
  /home/o0331dobe/Documents/AUR_packages/python-rsgislib/src/rsgislib-5.0.7/build-test/tests/test_regression_regresssklearn.py:112: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples,), for example using ravel().
    et_obj.fit(x, y)

tests/test_vectorgeoms.py::test_explode_vec_lyr
tests/test_vectorgeoms.py::test_explode_vec_files
  /home/o0331dobe/Documents/AUR_packages/python-rsgislib/src/rsgislib-5.0.7/build-test/rsgislib/vectorgeoms/__init__.py:1634: FutureWarning: Currently, index_parts defaults to True, but in the future, it will default to False to be consistent with Pandas. Use `index_parts=True` to keep the current behavior and True/False to silence the warning.
    data_explode_gdf = data_gdf.explode()

tests/test_vectorutils.py::test_perform_spatial_join_empty
tests/test_vectorutils.py::test_perform_spatial_join
  /usr/lib/python3.10/site-packages/_pytest/python.py:192: FutureWarning: The `op` parameter is deprecated and will be removed in a future release. Please use the `predicate` parameter instead.
    result = testfunction(**testargs)

tests/test_vectorutils.py::test_merge_vector_files
tests/test_vectorutils.py::test_merge_vector_files
tests/test_vectorutils.py::test_merge_vector_files
  /home/o0331dobe/Documents/AUR_packages/python-rsgislib/src/rsgislib-5.0.7/build-test/rsgislib/vectorutils/__init__.py:2711: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
    data_gdf = data_gdf.append(tmp_data_gdf)

tests/test_vectorutils.py::test_merge_utm_vecs_wgs84
tests/test_vectorutils.py::test_merge_utm_vecs_wgs84
  /home/o0331dobe/Documents/AUR_packages/python-rsgislib/src/rsgislib-5.0.7/build-test/rsgislib/vectorutils/__init__.py:3019: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
    out_gdf = out_gdf.append(data_gdf)

tests/test_vectorutils.py::test_merge_utm_vecs_wgs84
  /home/o0331dobe/Documents/AUR_packages/python-rsgislib/src/rsgislib-5.0.7/build-test/rsgislib/vectorutils/__init__.py:2852: ShapelyDeprecationWarning: Iteration over multi-part geometries is deprecated and will be removed in Shapely 2.0. Use the `geoms` property to access the constituent parts of a multi-part geometry.
    for poly in row["geometry"]:

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

---------- coverage: platform linux, python 3.10.4-final-0 -----------
Name                                              Stmts   Miss  Cover
---------------------------------------------------------------------
rsgislib/__init__.py                                283     86    70%
rsgislib/changedetect/__init__.py                     1      0   100%
rsgislib/changedetect/pxloutlierchng.py             170     95    44%
rsgislib/classification/__init__.py                 404    177    56%
rsgislib/classification/classaccuracymetrics.py     889     46    95%
rsgislib/classification/classimblearn.py            109    109     0%
rsgislib/classification/classkeraschips.py          187    187     0%
rsgislib/classification/classkeraspxl.py            155    155     0%
rsgislib/classification/classlightgbm.py            467    467     0%
rsgislib/classification/classratutils.py             62      0   100%
rsgislib/classification/classsklearn.py             263     14    95%
rsgislib/classification/classxgboost.py             691    691     0%
rsgislib/classification/clustersklearn.py           112     10    91%
rsgislib/dataaccess/__init__.py                       1      1     0%
rsgislib/dataaccess/usgs_m2m.py                     242    242     0%
rsgislib/elevation/__init__.py                       42      9    79%
rsgislib/imagecalc/__init__.py                      692    119    83%
rsgislib/imagecalc/calcindices.py                   133     13    90%
rsgislib/imagecalc/leastcostpath.py                  51     51     0%
rsgislib/imagecalc/specunmixing/__init__.py         606    302    50%
rsgislib/imagecalibration/__init__.py               115    115     0%
rsgislib/imagecalibration/sensorlvl2data.py          64     64     0%
rsgislib/imagecalibration/solarangles.py             56     56     0%
rsgislib/imagefilter/__init__.py                    188      0   100%
rsgislib/imagefilter/tiledfilter.py                 444    155    65%
rsgislib/imagemorphology/__init__.py                  2      0   100%
rsgislib/imageregistration/__init__.py               53     15    72%
rsgislib/imageutils/__init__.py                    1645    301    82%
rsgislib/imageutils/imagecomp.py                    513    513     0%
rsgislib/imageutils/imagelut.py                      99      9    91%
rsgislib/imageutils/stdimgblockiter.py              171     22    87%
rsgislib/imageutils/tilingutils.py                  109    109     0%
rsgislib/rastergis/__init__.py                      458    288    37%
rsgislib/regression/__init__.py                      41      3    93%
rsgislib/regression/regresssklearn.py               184     16    91%
rsgislib/segmentation/__init__.py                     2      0   100%
rsgislib/segmentation/shepherdseg.py                189     27    86%
rsgislib/segmentation/skimgseg.py                   251    251     0%
rsgislib/segmentation/tiledclump.py                 167     16    90%
rsgislib/segmentation/tiledsegsingle.py             252    252     0%
rsgislib/timeseries/__init__.py                       1      0   100%
rsgislib/timeseries/modelfitting.py                 195     24    88%
rsgislib/timeseries/robustfitoutliners.py           137    137     0%
rsgislib/timeseries/tmask.py                        102    102     0%
rsgislib/tools/__init__.py                            1      0   100%
rsgislib/tools/checkdatasets.py                     410    158    61%
rsgislib/tools/filetools.py                         556    257    54%
rsgislib/tools/ftptools.py                          209    189    10%
rsgislib/tools/geometrytools.py                     238     68    71%
rsgislib/tools/googlecloud.py                        26     26     0%
rsgislib/tools/httptools.py                          83     83     0%
rsgislib/tools/plotting.py                          425    232    45%
rsgislib/tools/projection.py                         57      4    93%
rsgislib/tools/sensors.py                            16     16     0%
rsgislib/tools/stats.py                             389    274    30%
rsgislib/tools/sysprofile.py                         45     45     0%
rsgislib/tools/testimages.py                         30     30     0%
rsgislib/tools/tilecacheutils.py                    187    134    28%
rsgislib/tools/utils.py                             359    170    53%
rsgislib/tools/utm.py                               209    120    43%
rsgislib/tools/visualisation.py                     791    509    36%
rsgislib/vectorattrs/__init__.py                    379     58    85%
rsgislib/vectorgeoms/__init__.py                   1738    371    79%
rsgislib/vectorutils/__init__.py                   1407    253    82%
rsgislib/vectorutils/createrasters.py                77     23    70%
rsgislib/vectorutils/createvectors.py               496    159    68%
rsgislib/zonalstats/__init__.py                    1320    287    78%
---------------------------------------------------------------------
TOTAL                                             20446   8715    57%

================================================================================================================================== short test summary info ==================================================================================================================================
FAILED tests/test_tools_ftptools.py::test_upload_ftp_file - ftplib.error_proto: /usr/sbin/vsftpd: Relink `/lib64/libnss_myhostname.so.2' with `/lib64/librt.so.1' for IFUNC symbol `clock_gettime'
FAILED tests/test_tools_ftptools.py::test_download_ftp_file - ftplib.error_proto: /usr/sbin/vsftpd: Relink `/lib64/libnss_myhostname.so.2' with `/lib64/librt.so.1' for IFUNC symbol `clock_gettime'
FAILED tests/test_tools_ftptools.py::test_download_curl_ftp_file - ftplib.error_proto: /usr/sbin/vsftpd: Relink `/lib64/libnss_myhostname.so.2' with `/lib64/librt.so.1' for IFUNC symbol `clock_gettime'
============================================================================================================ 3 failed, 809 passed, 72 skipped, 131 warnings in 327.20s (0:05:27) ============================================================================================================

morphological closing requirement

@petebunting Morphological closing provides an option to use a morph file. If set to False, a file still needs to be provided before the code will run. For example, this will not work:

output_img = in_img.replace('.tif','_beta0_dB_Wmask_clump_close.kea')
tmp_img = in_img.replace('.tif','_beta0_dB_Wmask_clump_close_tmp.kea')
morph_op_file = None
use_op_file = False
op_size = 5
gdalformat = "KEA"
datatype = rsgislib.TYPE_16UINT
niters = 1

while this will work:

output_img = in_img.replace('.tif','_beta0_dB_Wmask_clump_close.kea')
tmp_img = in_img.replace('.tif','_beta0_dB_Wmask_clump_close_tmp.kea')
morph_op_file = '/morph_closing.txt'
use_op_file = False
op_size = 5
gdalformat = "KEA"
datatype = rsgislib.TYPE_16UINT
niters = 1

probably just a catch needed on the use_op_file arg?

Divide by gamma?

Should you not be dividing by gamma instead of multiplying by it to match Shepherd's Equation 8?

rsgislib/src/calibration/RSGISApply6SCoefficients.cpp

Line 773 in 1a90429

output[i] = (LVal) / ((bandValues[irrDirBandIdx] * gamma) + (bandValues[irrDifandIdx] * this->brdfBeta));

Error when i use runShepherdSegmentation

input = r"xxx.tif"
output = r"xxxx.kea"

segutils.runShepherdSegmentation(input,output,gdalformat='KEA')

_segmentation.error: Could not create GDALDataset'

how can i fix this error,thank you!!!

Segmentation Runtime Error

Hi,
I was trying to run slic segmentation and got Runtime while doing so. Also, can we have segmentation with more than 3 bands ?

My code : -
perform_slic_segmentation( input_img, output_img, gdalformat='KEA', no_data_val=0, tmp_dir='tmp', calc_stats=True, n_segments=125000, compactness=0.1)

Error -
`---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
File :4, in

File C:\ProgramData\Anaconda3\envs\rsgis\lib\site-packages\rsgislib\segmentation\skimgseg.py:445, in perform_slic_segmentation(input_img, output_img, gdalformat, no_data_val, tmp_dir, calc_stats, use_pca, n_pca_bands, pca_pxl_sample, n_segments, compactness, max_iter, sigma, spacing, convert_to_lab, enforce_connectivity, min_size_factor, max_size_factor, slic_zero)
429 segResult = skimage.segmentation.slic(
430 pxlVals,
431 n_segments=n_segments,
(...)
441 slic_zero=slic_zero,
442 )
443 segResult = segResult + 1
--> 445 gdalOutDS = gdal.Open(output_img, gdal.GA_Update)
446 gdalOutBand = gdalOutDS.GetRasterBand(1)
447 gdalOutBand.WriteArray(segResult)

File C:\ProgramData\Anaconda3\envs\rsgis\lib\site-packages\osgeo\gdal.py:4580, in Open(*args)
4578 def Open(*args):
4579 """Open(char const * utf8_path, GDALAccess eAccess) -> Dataset"""
-> 4580 return _gdal.Open(*args)

RuntimeError: The HDF5ImageDataset driver does not support update access to existing datas`

Thanks for your help !

band_math docs

From the band_math docs, the example is:

band_defns = list()
band_defns.append(rsgislib.imagecalc.BandDefn('b1', 'img.kea', 1))
band_defns.append(rsgislib.imagecalc.BandDefn('b2', 'img.kea', 2))
rsgislib.imagecalc.band_math('out.kea', 'b1*b2', 'KEA', rsgislib.TYPE_32FLOAT, band_defns)

no imports are provided. Only really worth it here as the import for BandDefns should be explicitly stated. Suggest a move to:

import rsgislib
from rsgislib import imagecalc
from rsgislib.imagecalc import BandDefn

band_defns = list()
band_defns.append(rsgislib.imagecalc.BandDefn('b1', 'img.kea', 1))
band_defns.append(rsgislib.imagecalc.BandDefn('b2', 'img.kea', 2))
rsgislib.imagecalc.band_math('out.kea', 'b1*b2', 'KEA', rsgislib.TYPE_32FLOAT, band_defns)

May be worth keeping this ticket open for all documentation issues

Problems with calc_clear_sky_regions

It looks like the calc_clear_sky_regions function

rsgislib/python/rsgislib/imagecalibration/__init__.py

Line 142 in 2f61cfe

def calc_clear_sky_regions(

never quite made the migration over to rsgislib 5.0 fully. There is at least one case of incorrectly named keyword arguments being used in calls to other rsgislib functions (camelCase rather than snake_case), and at least one case of a missing argument.

We have a fix in progress, as soon as it is ready and tested we will open a PR. Work in progress here: https://github.com/EnvSys/rsgislib/blob/fix_clearsky/python/rsgislib/imagecalibration/__init__.py#L142

EDIT: Sorry, hit submit by accident too early

Pip compatibility

Hi folks - are there plans to make rsgislib pip compatible? Thanks!

Question for segment a particularly large remote sensing image

I would like to ask, if I want to segment a particularly large remote sensing image, how do I use rsgislib to segment the remote sensing image and there is no obvious boundary feature after merging

Segmentation module seems to be broken .

I have tried importing the segmentation module and got the following error

import rsgislib
from rsgislib.segmentation import shepherdseg

`---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
Input In [2], in <cell line: 15>()
13 # Import the RSGISLib modules for the notebook
14 import rsgislib
---> 15 from rsgislib.segmentation import shepherdseg
16 from rsgislib.segmentation.skimgseg import perform_slic_segmentation
17 from rsgislib.segmentation import tiledsegsingle

File D:\anaconda3\envs\geo\lib\site-packages\rsgislib\segmentation_init_.py:33, in
2 """
3 The segmentation module contains the segmentation functionality for RSGISLib.
4
(...)
29
30 """
32 # import the C++ extension into this level
---> 33 from ._segmentation import *

ImportError: DLL load failed while importing _segmentation: The specified module could not be found.
`

rasterization strategies in zonalstats funcs

Hi,

is there one of the zonalstats.METHOD_* that when applied by zonalstats.image_zone_to_hdf would yield same extraction results as the the other zonalstats functions?

Lately I have been trying out the zonalstats functions to extract hdf5 datasets and zonalstats.image_zone_to_hdf in particular seems to be quite fast. However, compared to the other functions of the zonalstats module and a rasterstats based implementation I wrote myself, the rasterization strategy seems to yield different results in number of pxls returned no matter which one I pick among the zonalstats.METHOD_* available.

The closest I got to the numbers returned by the other implementations is by setting
pxl_in_poly_method=zonalstats.METHOD_POLYOVERLAPSORCONTAINSPIXEL
and then masking out nodata values extracted on a masked raster by the same function, but still a bit short (50 something pxls over 15k so probably no big deal, but still wondering why). This is for extracting rois from polygons distanced enough so I wasn't concerned about extracting pixels multiple times by adopting a more generous strategy.

There are a few quite small polygons (that with rasterstats and other rsgislib functions yield less than 5 pxls), but even when removing these there is still some difference in the dimension of the retrieved datasets.