apolat2018/lsat

Landslide Susceptibility Assesment Tool

Landslide Susceptibility Assessment tool (LSAT)

LSAT scripts have been prepared for the assessment of landslide susceptibility.

LSAT includes ten python script. These are:

• Preparing_Data.py

• frequency_ratio.py

• information_value.py

• Logistic_Regression.py

• randomforest.py

• MLP.py

• tune_lr.py

• tune_rf.py

• tune_mlp.py

• Create_LSM&Calculate_ROC.py

  • LSAT tool file.

• "Preparing Data.py" prepares data to be used in analysis.

• The five scripts (frequency_ratio.py, information_value.py, Logistic_Regression.py, randomforest.py, and MLP.py) are used to create landslide susceptibility map with the methods of Frequency Ratio (FR), Information Value (IV), Logistic Regression (LR), Random Forest (RF), and Multi-Layer Perceptron (MLP).

• Also, this tool includes tuning scripts (tune_lr.py, tune_rf.py, tune_mlp.py) for the methods of LR, RF, and MLP.

• Create_LSM&Calculate_ROC.py is used to creates Landslide Susceptibility Map and calculates Area Under Curve (AUC) values with data including X-Y coordinate and probability fields. Prepared data using this script can be analyzed in external software. Then classification results can be processed with Create LSM and Calculate ROC script in GIS and susceptibility map can be created with AUC values.

Preparing reclassed landslide factor data

Firstly you should prepare reclassed landslide factor raster files before using the toolbox. They should be the same sizes and same resolutions. Prepared raster files should be as shown in the figure below.

IMPORTANT: All parameter files should be in the same folder. There should be no files other than factor raster files in this folder. The names of the parameters (factors) raster files must begin with "rec". rec_asp, rec_slp etc. Please do not use too long file names.

Landslide file must be polygonal type as shapefile format.
Also, the area file must be polygonal type as shapefile format.
The sample data folder is given below:
https://github.com/apolat2018/LSAT/tree/master/sample_data
Factor raster files are in raster folder. Landslides and area file are in vectors folder.

Required libraries

LSAT works on ArcGIS software with the Windows platform. Before using the LSAT, the required installations must be done. Python 2.7 is already installed with ArcGIS 10.4. The libraries must be compatible with the version of Python 2.7. Pip is recommended for easy installation. If a different version of python is installed, pip2.7 must be used. Scikitlearn (Pedregosa et al. 2011) is the main library for our tasks.

• Numpy (version of 1.8.2 or higher)
• SciPy (version of 0.13.3 or higher)
• Scikit-learn. Python 2.7 supports the versions of Scikit-learn 0.20 and earlier. In this study, the version of 0.20.2 was installed. Also, Numpy 1.15.4,
• Pandas 0.16.1
• Matplotlib 1.4.3
  In addition to these, the C++ compiler must be installed for windows.

How to install python libraries

• open file explorer
• find scripts folder in python 2.7
  • It is usually located in C:\Python27\ArcGISX.X\Scripts
    ArcGISX.X (X.X depends on the version of ArcGIS)
• open command prompt (cmd)
  • Click on the address bar in the file explorer. Now type cmd in the address bar and press enter. Command Prompt will run.
• You should in "C:\Python27\ArcGISX.X\Scripts"
  • Now you are ready to install python 2.7 packages
  • You can use pip as "pip install package name==package version"
    • pip install numpy
    • pip install scipy
    • pip install scikit-learn==0.20.2
    • pip install pandas
    • pip install matplotlib

How to use the LSAT toolbox

• After the installations are done, download the toolbox and script files with py extensions to same folder.
• Open ArcGis.
• Go to Catalog and open the toolbox file in the downloaded folder.
• Please check file location for all scripts.To do this:
  • Right click on script
  • Select properties
  • Select search
  • choose script file
  • Do this for all script files in toolbox.

  

Preparing data for analysis

• Double click "1- Data Preparation" script to prepare data.

  • Select the folder of landslide parameters (The name of the parameter raster files must begin with "rec". rec_aspect, rec_slope etc.)
  • Select landslide shp file (Landslides file(.shp) must be polygon type)
  • Select area file (must be polygonal type .shp)
  • Select cell size
  • Select Train-Validation Split size (%). 70 mean %70 of data for train and 30% of data for test
  • Click "OK"

  

Analysis

Choose an analysis method.
For example, if you chose logistic regression:

• Double click "5- Logistic Regression method"
• Select workspace folder (The folder including output data of Data Preparation script)
• Select save file folder (Susceptibility map and ROC graph is saving this Folder)
• Select Coordinate system
• Select cell size
• Select weighting data type frequency ratio or information value
• Select C value: Inverse of regularization strength; must be a positive float. Like in support vector machines, smaller values specify stronger regularization.
• Select maximum iteration value:Maximum number of iterations taken for the solvers to converge.
• Select solver:'newton-cg', 'lbfgs', 'liblinear', 'sag', 'saga'

This script will create landslide susceptiblity map with logistic regression method.
You can choose the methods of Frequency Ratio (FR), Information Value (IV), Logistic Regression (LR), Random Forest (RF), and Multi-Layer Perceptron (MLP) with the LSAT tool.

If you have an excel file including x, y coordinate values, and probability fields you can use the script of "2- Create LSM and Calculate ROC". The excel file can be created by external software with different analysis methods. If you want to use the analysis methods of the LSAT tool you don't need to use this script.

Supplementary
Annex
Figures

Dr. Ali POLAT (2021)
[email protected]

Cite this article

Polat, A. An innovative, fast method for landslide susceptibility mapping using GIS-based LSAT toolbox. Environ Earth Sci 80, 217 (2021). https://doi.org/10.1007/s12665-021-09511-y

lsat's Issues

Random Forest Performance Calculation Error

@apolat2018 Random Forest model is able to train but an error pops up during the performance calculation. The error is as follows:

Starting Training
Saving Prediction Data as rf.csv
Analysis finished
Creating SUSCEPTIBILITY Map and Calculating ROC
Susceptibility Map was created in D:\LSM\Results\RF folder as rf_sus raster file
Failed script RF...

Traceback (most recent call last):
File "C:\Python27\ArcGIS10.4\Scripts\LSAT-master\randomforest.py", line 145, in
trn.save(train_exp)
RuntimeError: ERROR 000871: D:\LSM\4_ArcGIS\train.tif: Unable to delete the output ????????????????????.

Failed to execute (RF).
Failed at Wed Jun 15 01:22:31 2022 (Elapsed Time: 6 minutes 30 seconds)

Error to execute the Preparing_Data.py file

I tried in your sample data. But it throws the following error.

Traceback (most recent call last):
  File "C:\Users\tek\Downloads\LSAT-master\Preparing_Data.py", line 61, in <module>
    arcpy.SubsetFeatures_ga(hey,"train_1.shp","test_1.shp",split_size,"PERCENTAGE_OF_INPUT")###Splitting landslides as %70 train and %30 test
  File "c:\program files (x86)\arcgis\desktop10.8\arcpy\arcpy\ga.py", line 1286, in SubsetFeatures
    raise e
ExecuteError: Failed to execute. Parameters are not valid.
ERROR 000824: The tool is not licensed.
Failed to execute (SubsetFeatures).


Failed to execute (prepare).

MLP Memory Error

@apolat2018 I get the following error when I run the MLP model:
Running script MLP...
Hidden layer size:---------------:100
Activation function:-------------:logistic
Solver:--------------------------:adam
Alpha----------------------------:0.0001
Learning rate:-------------------:constant
Learning rate init---------------:0.001
Max iter:------------------------:200
Momentum-------------------------:0.9
Starting analysis with MLP algorithm
Starting Training
Failed script MLP...

Traceback (most recent call last):
File "C:\Python27\ArcGIS10.4\Scripts\LSAT-master\MLP.py", line 109, in
tahmin=mlp.predict_proba(pre)
File "C:\Python27\ArcGIS10.4\lib\site-packages\sklearn\neural_network\multilayer_perceptron.py", line 1054, in predict_proba
y_pred = self._predict(X)
File "C:\Python27\ArcGIS10.4\lib\site-packages\sklearn\neural_network\multilayer_perceptron.py", line 674, in _predict
layer_units[i + 1])))
MemoryError

Failed to execute (MLP).
Failed at Wed Jun 15 02:58:11 2022 (Elapsed Time: 1 minutes 13 seconds)

error in preparing data

hi can you help me fix this issue :

Traceback (most recent call last):
File "C:\Users\This PC\Videos\LSAT\Preparing_Data.py", line 97, in
arcpy.SubsetFeatures_ga("erased_poi.shp","data_trn.shp","",no_ls_trn,"ABSOLUTE_VALUE")###random no landslide pixels is selecting as many as train landslide pixels
File "c:\program files (x86)\arcgis\desktop10.8\arcpy\arcpy\ga.py", line 1286, in SubsetFeatures
raise e
ExecuteError: ERROR 040303: The input features, the output training feature class, or the output test feature class have no records.
Failed to execute (SubsetFeatures).

Failed to execute (prepare).

Error

Please help me find the error

Executing: prepare D:\Desktop\SUS "D:\Desktop\MM NAB FINAL\BJ_MM_FINAL_Merge.shp" D:\Desktop\Nabatiye_Data\StudyArea.shp 150 70
Start Time: Sun Jun 6 20:08:14 2021
Running script prepare...
[u'rec_aspect.tif', u'rec_elev.tif', u'rec_EQ.tif', u'rec_Faults1.tif', u'rec_flood.tif', u'rec_Geology.tif', u'rec_LUC.tif', u'rec_Rain.tif', u'rec_River1.tif', u'rec_Roads1.tif', u'rec_slope.tif', u'rec_soil.tif']
Starting DATA PREPARATION...
12 raster data imported as parameter
Failed script prepare...

Traceback (most recent call last):
File "C:\Python27\ArcGIS10.8\Scripts\Preparing_Data.py", line 35, in
arcpy.Clip_management(i,"",clipped,area,"","ClippingGeometry","MAINTAIN_EXTENT")
File "c:\program files (x86)\arcgis\desktop10.8\arcpy\arcpy\management.py", line 15973, in Clip
raise e
ExecuteError: ERROR 999999: Error executing function.
Configuration RasterCommander.ImageServer can not be started.
Server Object instance creation failed on machine BILAL-MSI.
Image Server is not supported with the current license.
Failed to execute (Clip).

Failed to execute (prepare).
Failed at Sun Jun 6 20:08:29 2021 (Elapsed Time: 15.37 seconds)

ERROR

help me please

Executing: prepare C:\Users\palul\Downloads\SKRIPSI\RASTER C:\Users\palul\Downloads\SKRIPSI\VECTOR\14.shp C:\Users\palul\Downloads\SKRIPSI\VECTOR\IDN_9_24_1.shp 25 70
Start Time: Mon Dec 12 09:58:58 2022
Running script prepare...
[u'rec_asp', u'rec_curv', u'rec_elev', u'rec_lsf', u'rec_ndvi', u'rec_slp', u'rec_tri', u'rec_twi']
Starting DATA PREPARATION...
8 raster data imported as parameter
Area is converting to Raster
Raster is converting to Point
Failed script prepare...

Traceback (most recent call last):
File "C:\Users\palul\Downloads\LSAT\Preparing_Data.py", line 46, in
all_poi=arcpy.RasterToPoint_conversion(hepsi,"all_poi.shp","VALUE")
File "c:\program files (x86)\arcgis\desktop10.6\arcpy\arcpy\conversion.py", line 338, in RasterToPoint
raise e
ExecuteError: ERROR 010151: No features were written to the feature class.
Failed to execute (RasterToPoint).

Failed to execute (prepare).
Failed at Mon Dec 12 10:01:43 2022 (Elapsed Time: 2 minutes 45 seconds)

ERROR in IV method

Executing: IV F:\LSAT-master\LSAT-master\Data\Raster_New F:\LSAT-master\LSAT-master\Data\Information_Value PROJCS['WGS_1984_UTM_Zone_46N',GEOGCS['GCS_WGS_1984',DATUM['D_WGS_1984',SPHEROID['WGS_1984',6378137.0,298.257223563]],PRIMEM['Greenwich',0.0],UNIT['Degree',0.0174532925199433]],PROJECTION['Transverse_Mercator'],PARAMETER['False_Easting',500000.0],PARAMETER['False_Northing',0.0],PARAMETER['Central_Meridian',93.0],PARAMETER['Scale_Factor',0.9996],PARAMETER['Latitude_Of_Origin',0.0],UNIT['Meter',1.0]] 30
Start Time: Mon Mar 18 23:34:44 2024
Running script IV...
Starting Information Value Analysis...
Analysis finished
Susceptibility Map was created in F:\LSAT-master\LSAT-master\Data\Information_Value folder as iv_sus raster file
ROC calculation is starting.....
Success rate is: 93.8828051661
prediction rate is: 95.4902950693
Failed script IV...

Traceback (most recent call last):
File "F:\LSAT-master\LSAT-master\information_value.py", line 336, in
sc=plt.plot(tot,tr,color="red",label=":Success Rate"+"("+str(f)+")")
File "C:\Python27\ArcGIS10.8\lib\site-packages\matplotlib\pyplot.py", line 3352, in plot
ax = gca()
File "C:\Python27\ArcGIS10.8\lib\site-packages\matplotlib\pyplot.py", line 969, in gca
return gcf().gca(**kwargs)
File "C:\Python27\ArcGIS10.8\lib\site-packages\matplotlib\pyplot.py", line 586, in gcf
return figure()
File "C:\Python27\ArcGIS10.8\lib\site-packages\matplotlib\pyplot.py", line 533, in figure
**kwargs)
File "C:\Python27\ArcGIS10.8\lib\site-packages\matplotlib\backend_bases.py", line 158, in new_figure_manager
from matplotlib.figure import Figure
File "C:\Python27\ArcGIS10.8\lib\site-packages\matplotlib\figure.py", line 40, in
from matplotlib.axes import Axes, SubplotBase, subplot_class_factory
File "C:\Python27\ArcGIS10.8\lib\site-packages\matplotlib\axes_init_.py", line 4, in
from ._subplots import *
File "C:\Python27\ArcGIS10.8\lib\site-packages\matplotlib\axes_subplots.py", line 10, in
from matplotlib.axes._axes import Axes
File "C:\Python27\ArcGIS10.8\lib\site-packages\matplotlib\axes_axes.py", line 33, in
import matplotlib.quiver as mquiver
File "C:\Python27\ArcGIS10.8\lib\site-packages\matplotlib\quiver.py", line 171, in
""" % docstring.interpd.params
KeyError: u'PolyCollection'

Failed to execute (IV).
Failed at Mon Mar 18 23:35:15 2024 (Elapsed Time: 31.62 seconds)

ROC Plot fails for Frequency Ratio Method

@apolat2018 I get the following error during the generation of the ROC Curve for the Frequency Ratio Method. The last but one value is a scientific notation (8.00640512409e-05); I am not sure if that's what caused the error, and I have no control over the generation of these values. How do I rectify this problem?

Start Time: Tue Jun 07 14:18:36 2022
Running script FRanalysis...
Analysis started
Starting Frequency Ratio Analysis...
Analysis finished
Susceptibility Map was created in D:\LSM\Results\FR folder as fr_sus raster file
ROC calculation is starting.....
Success rate is: 86.963695546
prediction rate is: 86.6727031059
101
[1, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.999973311983, 0.999973311983, 0.999973311983, 0.999866559915, 0.999786495863, 0.999706431812, 0.99954630371, 0.999386175607, 0.999092607419, 0.998799039231, 0.998638911129, 0.998238590873, 0.997331198292, 0.996637309848, 0.995623165199, 0.994875900721, 0.993408059781, 0.99186015479, 0.990258873766, 0.988150520416, 0.985641846811, 0.982706164932, 0.979049906592, 0.975366960235, 0.971897518014, 0.967333867094, 0.962022951695, 0.955084067254, 0.947371230318, 0.939097945023, 0.930504403523, 0.921910862023, 0.912836936216, 0.900960768615, 0.88820389645, 0.87440619162, 0.859674406192, 0.844408860422, 0.825753936483, 0.805684547638, 0.783773685615, 0.761649319456, 0.74040565786, 0.71654657059, 0.691619962637, 0.663090472378, 0.636055511076, 0.606805444355, 0.576354416867, 0.548011742728, 0.518361355751, 0.487536696023, 0.453135842007, 0.418895116093, 0.384947958367, 0.353696290366, 0.323725647184, 0.295276220977, 0.268027755538, 0.237176407793, 0.209153989859, 0.184307445957, 0.163090472378, 0.140245529757, 0.11956231652, 0.100373632239, 0.0818254603683, 0.0645049372832, 0.0506805444355, 0.038217240459, 0.0259941286362, 0.0178542834267, 0.0111022151054, 0.00725914064585, 0.00416333066453, 0.00232185748599, 0.00130771283694, 0.000560448358687, 0.000266880170803, 8.00640512409e-05, 0.0]
Failed script FRanalysis...

Traceback (most recent call last):
File "C:\Python27\ArcGIS10.4\Scripts\LSAT-master\frequency_ratio.py", line 340, in
plt.plot(tot,tr,color="red",label=":Success Rate"+"("+str(f)+")")#success rate
File "C:\Python27\ArcGIS10.4\lib\site-packages\matplotlib\pyplot.py", line 3092, in plot
ax = gca()
File "C:\Python27\ArcGIS10.4\lib\site-packages\matplotlib\pyplot.py", line 828, in gca
ax = gcf().gca(**kwargs)
File "C:\Python27\ArcGIS10.4\lib\site-packages\matplotlib\figure.py", line 1274, in gca
return self.add_subplot(1, 1, 1, **kwargs)
File "C:\Python27\ArcGIS10.4\lib\site-packages\matplotlib\figure.py", line 937, in add_subplot
if isinstance(args[0], SubplotBase):
TypeError: isinstance() arg 2 must be a class, type, or tuple of classes and types

Failed to execute (FRanalysis).
Failed at Tue Jun 07 14:18:51 2022 (Elapsed Time: 15.12 seconds)

Error in Logistic regression method

Dear Sir,

I prepared the sample data you provided and ran logistic regression method but it showed this error.

Traceback (most recent call last):
  File "C:\LSAT-master\LSAT-master\Logistic_Regression.py", line 14, in <module>
    from sklearn.linear_model import LogisticRegression
  File "C:\Python27\ArcGIS10.8\lib\site-packages\sklearn\__init__.py", line 64, in <module>
    from .base import clone
  File "C:\Python27\ArcGIS10.8\lib\site-packages\sklearn\base.py", line 13, in <module>
    from .utils.fixes import signature
  File "C:\Python27\ArcGIS10.8\lib\site-packages\sklearn\utils\__init__.py", line 12, in <module>
    from .murmurhash import murmurhash3_32
  File "__init__.pxd", line 164, in init sklearn.utils.murmurhash
ValueError: numpy.dtype has the wrong size, try recompiling. Expected 52, got 56

Failed to execute (LOGr).
Failed at Mon Mar 21 21:31:00 2022 (Elapsed Time: 0.12 seconds)

Error

Please what is the error

Executing: prepare D:\Desktop\sss "D:\Desktop\MM NAB FINAL\BJ_MM_FINAL_Merge.shp" D:\Desktop\Nabatiye_Data\StudyArea.shp 150 70
Start Time: Sun Jun 6 20:32:50 2021
Running script prepare...
[u'rec_EQ.tif', u'rec_Faults1.tif', u'rec_flood.tif', u'rec_Geology.tif', u'rec_LUC.tif', u'rec_Rain.tif', u'rec_River1.tif', u'rec_Roads1.tif', u'rec_soil.tif']
Starting DATA PREPARATION...
9 raster data imported as parameter
Area is converting to Raster
Raster is converting to Point
PREPARING TRAIN AND VALIDATION DATA
Clipping landslides
landslide pixels is converting to point
Failed script prepare...

Traceback (most recent call last):
File "C:\Python27\ArcGIS10.8\Scripts\Preparing_Data.py", line 97, in
arcpy.SubsetFeatures_ga("erased_poi.shp","data_trn.shp","",no_ls_trn,"ABSOLUTE_VALUE")###random no landslide pixels is selecting as many as train landslide pixels
File "c:\program files (x86)\arcgis\desktop10.8\arcpy\arcpy\ga.py", line 1286, in SubsetFeatures
raise e
ExecuteError: Failed to execute. Parameters are not valid.
ERROR 000854: The value is out of the range from 1 to 2.14748e+09.
Failed to execute (SubsetFeatures).

Failed to execute (prepare).
Failed at Sun Jun 6 20:33:38 2021 (Elapsed Time: 47.14 seconds)