jbferet/biodivmapr

biodivMapR: an R package for α- and β-diversity mapping using remotely-sensed images

biodivMapR

An R package for α- and β-diversity mapping using remotely-sensed images

1 Install

The package remotes first needs to be installed from the CRAN

install.packages("remotes")

Then some packages which were removed from the CRAN may need to be installed directly from authors' repository. This is the case for dissUtils:

remotes::install_github('cran/dissUtils')

After installing remotesand dissUtils, biodivMapR should be ready for installation with the following command line in your R session:

remotes::install_github('jbferet/biodivMapR')

2 Tutorial

A tutorial vignette is available here.

The corresponding script is available in file examples/tutorial.R.

3 Citation

If you use biodivMapR, please cite the following references:

Féret, J.-B., de Boissieu, F., 2019. biodivMapR: an R package for α‐ and β‐diversity mapping using remotely‐sensed images. Methods Ecol. Evol. 00:1-7. https://doi.org/10.1111/2041-210X.13310

Féret, J.-B., Asner, G.P., 2014. Mapping tropical forest canopy diversity using high-fidelity imaging spectroscopy. Ecol. Appl. 24, 1289–1296. https://doi.org/10.1890/13-1824.1

4 Acknowledgments / Fundings

This research was supported by the Agence Nationale de la Recherche (ANR, France) through the young researchers project BioCop (ANR-17-CE32-0001)

biodivmapr's Issues

line 133-137

Sel_PC <- select_PCA_components(Input_Image_File = Input_Image_File,
Output_Dir = Output_Dir,
PCA_Files = PCA_Output$PCA_Files,
TypePCA = PCA_Output$TypePCA,
File_Open = TRUE)
How can I choose the path? Which path is appropriate?

raster import

I am trying to import a 5-band raster and run the demo code on it. It works through the call to perform_PCA() call, and just hangs there. perform_radiometric_filtering() seems to work correctly. I am attaching a debugging version of the code and a small cropped area from my raster. I am using convert2bil() and add the correct wavelength info for the sensor (Micasense RedEdge) and I have tried multiple different versions of raster grids, etc., but none of it seems to make a difference. Here I am providing a GeoTiff. Thanks in advance.

> print("PERFORM DIMENSIONALITY REDUCTION")
> PCA_Output <- perform_PCA(Input_Image_File = Input_Image_File, Input_Mask_File = Input_Mask_File,
>                           Output_Dir = Output_Dir, TypePCA = TypePCA, FilterPCA=FilterPCA,
>                           nbCPU = nbCPU, MaxRAM = MaxRAM, Continuum_Removal = Continuum_Removal)
> # *********************************************************
> #   IF MULTI / HYPERSPECTRAL DATA: 
> #   Please make sure the wavelengths are in nanometers or micrometers
> # if not, stop processing and convert wavelengths in nanometers
> # or micrometers in HDR file
> # *********************************************************
> #   Exclude spectral domains corresponding to water vapor absorption
> # The following spectral domains (min and max spectral band in nanometers) will be discarded
> # [,1] [,2]
> # Excluded_WL    0  400
> # 895 1005
> # 1180 1480
> # 1780 2040
> # Please define the input variable Excluded_WL when calling perform_PCA
> # if you want to modify these spectral domains
> # [1] "Extract pixels from the images to perform PCA on a subset"

code hangs here. I have let it run overnight and it never completes
go_debug.zip

UPDATE: I updated to R 4.0.2 and it seemed to run past the PCA. But the rasters it producing are sparse so there are still problems.

Processing data with higher resolution

Hi,
I'd like to process a PlanetScope image (resolution of 3x3m). However, I cannot calculate alpha and beta diversity.
For alpha, it starts mapping but after a while the
Error cannot allocate vector of size 10.9 Gb occurs.

Is there a way to process PlanetScope data other than processing little parts of it?

Thank you!

NAs for simpson index

Dear biodivmapR developers,

I first came across the issue of getting a map with only 0 as an output of the map_alpha_div for the Simpson index, when running code on my own dataset. What I found strange is that the function output for the Shannon index was resonable, where it seems to me that both index should have been computed from the same baseline data (i.e. I expect to have strange value for both or for none but not for only one of the index output).

When running your example code with the addition of the Simpson index in the Index_Alpha parameter of the map_alpha_div function I got only NA for the in the Simpson_10 output file, while again a resonable looking output for the Shannon index.

Is getting NA or 0 for Simpson index a reproducible problem to anyone?
If so, can we still trust the output for the Shannon index?

Best,

Marie-Lou

Unable to read .hdr file generated after using raster2BIL function

Hi. I am trying to go through your Tutorial for biodivMapR. I have a multiband (6 bands) sentinel-2 image (Ras_tif11.tif) which I have converted to ENVI HDR format using raster2BIL function. I have used the following code:

library(biodivMapR)

raster2BIL(
  "D:\\Data\\Ras_tif11.tif",
  Sensor = "unknown",
  Output_Dir = FALSE,
  Convert_Integer = TRUE,
  Multiplying_Factor = 1,
  Multiplying_Factor_Last = 1,
  Mask = FALSE
)

After running the above command I am getting following message in the console:

The converted file will be written in the following location:
[1] "D:\\Data\\biodivMapR_Convert_BIL\\Ras_tif11"
reading initial file
writing converted file
please make sure that the following header file contains information required
[1] "D:\\Data\\biodivMapR_Convert_BIL\\Ras_tif11.hdr"
or manually add wavelength location in HDR file, if relevant
[1] "D:\\Data\\biodivMapR_Convert_BIL\\Ras_tif11"

Post this when we are trying to "PERFORM RADIOMETRIC FILTERING" we are getting following error:

*********************************************************
No wavelength is associated to the image in the .hdr file
Please add wavelengths to the .hdr file
if you are processing multi or hyperspectral optical data
 Otherwise, make sure Continuum_Removal is set to FALSE  
*********************************************************
[1] "Create mask based on NDVI, NIR and Blue threshold"
wavelength units not provided in the header of the image
assuming wavelengths are expressed in micrometers
Error in h(simpleError(msg, call)) : 
  error in evaluating the argument 'x' in selecting a method for function 't': Cannot create a RasterLayer object from this file.
In addition: Warning message:
In max(HDR$wavelength) : no non-missing arguments to max; returning -Inf

Kindly help us out in resolving this issue.

Error in perform_PCA/ Issue with data formatting

When I perform the dimensionality reduction, as shown here:

PCA_Output <- perform_PCA(Input_Image_File = Input_Image_File, 
                          Input_Mask_File = Input_Mask_File,
                          Output_Dir = Output_Dir, 
                          TypePCA = TypePCA, 
                          FilterPCA = FilterPCA,
                          nbCPU = nbCPU, 
                          MaxRAM = MaxRAM, 
                          Continuum_Removal = Continuum_Removal)

I get the following error:

[1] "Extract pixels from the images to perform PCA on a subset"
Error in `[[<-.data.frame`(`*tmp*`, "sampleIndex", value = 1:0) : 
  replacement has 2 rows, data has 0

When I try to run the code with the test image provided, it works. So, I assume the problem is with the way the data is formatted. I tried both with ENVI and .tif file formats but the error persists. Do you have any tips and tricks for formatting the data?

Here is a subset of the Sentinel-2 tile I have used, together with the .hdr file: https://megastore.rz.uni-augsburg.de/get/9jgRjuA97J/

I use R version 4.3.3, biodivMapR version 1.14.0.

Processing time for large Sentinel-2 images

Hello,

I am working with tile-sized Sentinel-2 images that are around 1.7Gb in size (GeoTIFF format). I proceeded according to the tutorial and got to the PCA and dimensionality reduction section, then ran the perform_PCA function. This has now been running for the last 6 hours and is still on the part [1] "Extract pixels from the images to perform PCA on a subset". As I have only little experience with large remote sensed images, I was wondering if this is an acceptable amount of time or if I may have done something wrong?

I hope that this is the right place to ask this!

Thanks in advance,
Nikolas

Double forward slash in *select_PCA_components()* function

Hello,

When running the select_PCA_components() function I got the following error:
Error in editor(file = file, title = title) : argument "name" is missing, with no default
After checking the code I found an additional "/" in the Sel_PC resulting from the additional
"/".

The path was as follows: "C:/Users/.../S2A_T33NUD_20180104_Subset/SPCA//PCA/Selected_Components.txt"
Changing the following line of code:
Sel_PC <- paster(Output_Dir_Full, "/PCA/Selected_Components.txt", sep = "".
to:
Sel_PC <- paster(Output_Dir_Full, "PCA/Selected_Components.txt", sep = "".
resolved the error.

There is already a forward slash in:
Output_Dir_Full <- paste(Output_Dir, "/", Image_Name, "/", TypePCA, "/", sep = "")

I hope my explanations are clear.

Thank you for this package! I am currently looking through the code and read the papers!
This is amazing!

Best,
Niamh

line 144-149

In line 144-149，which is
Kmeans_info <- map_spectral_species(Input_Image_File = Input_Image_File,
Input_Mask_File = PCA_Output$MaskPath,
Output_Dir = Output_Dir,
SpectralSpace_Output = PCA_Output,
nbclusters = nbclusters,
nbCPU = nbCPU, MaxRAM = MaxRAM)
I met an error:
Error in Subset$DataSubset[, PC_Select] :
no 'dimnames' attribute for array
How to solve this problem?

Error Example_script from master Branch

Hello, First of all, thanks for sharing the code its really amazing, I was wondering If you could help me with my issue.
I am running example_script.R but when is calculating PCA_Output I get an error:

Exclude spectral domains corresponding to water vapor absorption
The following spectral domains (min and max spectral band in nanometers) will be discarded
            [,1] [,2]
Excluded_WL    0  400
             895 1005
            1180 1480
            1780 2040
Please define the input variable Excluded_WL when calling perform_PCA
if you want to modify these spectral domains
[1] "Extract pixels from the images to perform PCA on a subset"
[1] "perform PCA#1 on the subset image"
[1] "Apply PCA model to the whole image"
[1] "PCA Piece # 1 / 1"
Error in .rasterObjectFromFile(x, objecttype = "RasterBrick", ...) : 
  Cannot create a RasterLayer object from this file.
> 
> PCA_Files <- PCA_Output$PCA_Files
Error: object 'PCA_Output' not found

After this everything else fails.

Could you guide me, what I am doing wrong?
I follow this guide also
https://jbferet.github.io/biodivMapR/reference/perform_PCA.html

and the difference between (in the tutorial but not in the example) them is:
Excluded_WL <- c(0, 400)
Excluded_WL <- rbind(Excluded_WL, c(895, 1005))
Excluded_WL <- rbind(Excluded_WL, c(1180, 1480))
Excluded_WL <- rbind(Excluded_WL, c(1780, 2040))

Thanks!!

Computation of the alpha and beta diversity still uses the raw image file

1. According to the tutorial website, in order to compute the alpha and beta diversity, the raster image file generated in the step of Spectral Species Mapping should be used as input image file for the functions "map_alpha_div" and "map_beta_div". But in the tutorial example code, all these functions use the raw Sentinal-2 raster file as image input.

2. The Spectral Species Mapping step directly uses the result of PCA, instead of using the result of both PCA and component selection (some bands being discarded). However Functional Diversity this step takes into consideration of both PCA and component.

I apologize if the answers to these questions are already explicit.

Edit: grammar correction

A bug from function "perform_PCA"

Dr. Féret,
Thanks for your package. This is very exciting package so that I can use it to calculate beta diversity for my hyperspectral data. I am using a airborne hyperspectral data with 64 bands.
Here is my header file:

ENVI
samples = 1342
lines = 13596
bands = 64
header offset = 0
file type = ENVI Standard
data type = 2
data ignore value=-Inf
interleave = BIL
sensor type =Unknown
byte order = 0
wavelength units = Nanometers
map info = {projection, 1, 1, xxxx, xxx , 1 , 1 }
wavelength = { 398.55, 407.39, 416.27, 425.18, 434.13, 443.10,
452.11, 461.14, 4721, 479.30, 488.43, 497.58, 506.76, 515.96,
525.19, 534.44, 543.72, 553.02, 562.35, 571.70, 581.07, 5946,
599.86, 609.29, 618.74, 628.21, 637.69, 647.19, 656.70, 666.24,
675.78, 685.34, 694.91, 704.50, 714.09, 723.70, 733.32, 742.95,
752.58, 762.23, 771.88, 781.54, 791.21, 800.88, 810.56, 820.24,
829.92, 839.61, 849.30, 858.99, 868.68, 878.38, 888.07, 897.76,
907.45, 917.13, 926.81, 936.49, 946.16, 955.83, 965.49, 975.15,
984.80, 994.44}
coordinate system string = {PROJCS["WGS_1984_UTM_Zone_50N",GEOGCS["GCS_WGS_1984",DATUM....}
z plot range = { ...... }
band names = { ..... }

I encountered an issue when using function "perform_PCA".
If I set "Continuum_Removal = T", there is error message from function "future_xapply" in "apply_continuum_removal":

---

      Format of main raster OK for processing         

---

---

     Format of mask raster OK for processing         

---

[1] "Extract pixels from the images to perform PCA on a subset"
Error in Lambda_tmp - Latest.Intercept_tmp : non-conformable arrays

If I set "Continuum_Removal = F", there is error message from function "write_PCA_raster":

---

      Format of main raster OK for processing         

---

---

     Format of mask raster OK for processing         

---

[1] "Extract pixels from the images to perform PCA on a subset"
[1] "perform PCA#1 on the subset image"
[1] "perform 2nd filtering: Exclude extreme PCA values"
[1] "Perform PCA on image subsets and filter data"
[1] "PCA Piece # 1 / 11"
[1] "PCA Piece # 2 / 11"
[1] "PCA Piece # 3 / 11"
[1] "PCA Piece # 4 / 11"
[1] "PCA Piece # 5 / 11"
[1] "PCA Piece # 6 / 11"
[1] "PCA Piece # 7 / 11"
[1] "PCA Piece # 8 / 11"
[1] "PCA Piece # 9 / 11"
[1] "PCA Piece # 10 / 11"
[1] "PCA Piece # 11 / 11"
[1] "perform PCA#2 on the subset image"
[1] "Apply PCA model to the whole image"
[1] "PCA Piece # 1 / 11"
Error in t(PCA_model$eiV[, 1:Nb_PCs]) %*% t(center_reduce(Image_Chunk, :
non-conformable arguments.

Would you give me some suggestions to correct this error? Thanks in advance.
Best,
Xiangcheng

Error in example_script.R

Hi there,

I just got started with this package. I am trying to execute the example script but I get this error:

Error in Summary.factor(c(1L, 1L, 1L), na.rm = FALSE) : 
  ‘max’ not meaningful for factors
In addition: There were 50 or more warnings (use warnings() to see the first 50)

I am using the example_script.R from the package. I changed only the Output_Dir path and I have chosen PC 1, PC 2, PC 3 when asked to. Isn't the script suppose to work as it is?

Thank you in advance

+/- inf values in OutputPCA_8_PCs raster

Hello,

I am having an issue where the output raster of the PCA (file named OutputPCA_8_PCs) contains many +/- inf values in each band. I am using the package sen2r to download Sentinel-2 data, which is in ENVI format. What may cause the PCA to produce such values and how can I solve it? I can supply more precise information if it is helpful.

Thank you very much in advance!

Best,
Nikolas

Install error

Hi, thanks for making this amazing package - really useful! When installing I get an error that says it's not compatible with R version 4.0.2 or 4.0.3. Could you please let me know which version of R I should use it with? Thanks

Dependencies on rgdal and rgeos

I've heard of this package during a talk in Sageo conf.
How dependent is it on rgdal and rgeos?
These packages will retire soon.
CRAN packages have received these kind of warning :
riatelab/cartography#85

Detailed post on this matter: Upcoming changes to popular R packages for spatial data: what you need to do

You may be well aware of these changes, if so I am sorry to bother you with this.

cheers,

T.

dissUtils not available for biodivMapR

ERROR: dependency 'dissUtils' is not available for package 'biodivMapR'
.
.
.
.
.installed various versions of R, and still get this error.
even tried with dependencies=TRUE and FALSE, repos=NULL

Error during executing perform_PCA

Dr. Féret,
Thanks for your package. It's very exciting. There's one problem that's been bugging me for a long time. When I ran your example_script, It work well. But when I tried to import a my own raster and ran the demo code on it I got the following error:
PCA_Output <- perform_PCA(Input_Image_File = Input_Image_File, Input_Mask_File = Input_Mask_File,

•                       Output_Dir = Output_Dir, TypePCA = TypePCA, FilterPCA=FilterPCA,
  

•                       nbCPU = nbCPU, MaxRAM = MaxRAM, Continuum_Removal = Continuum_Removal)
  

Exclude spectral domains corresponding to water vapor absorption
The following spectral domains (min and max spectral band in nanometers) will be discarded
[,1] [,2]
Excluded_WL 0 400
895 1005
1320 1480
1780 2040
Please define the input variable Excluded_WL when calling perform_PCA
if you want to modify these spectral domains
[1] "Extract pixels from the images to perform PCA on a subset"
Error in CPL_read_gdal(as.character(x), as.character(options), as.character(driver), :
negative length vectors are not allowed
In addition: Warning messages:
1: In as.data.table.list(x, keep.rownames = keep.rownames, check.names = check.names, :
Item 1 has 0 rows but longest item has 2; filled with NA
2: In as.data.table.list(x, keep.rownames = keep.rownames, check.names = check.names, :
Item 2 has 0 rows but longest item has 2; filled with NA
3: In min(x) : no non-missing arguments to min; returning Inf
4: In max(x) : no non-missing arguments to max; returning -Inf
5: In max(rc$col) : no non-missing arguments to max; returning -Inf
6: In min(rc$rowInBlock) : no non-missing arguments to min; returning Inf
7: In CPL_read_gdal(as.character(x), as.character(options), as.character(driver), :
NAs introduced by coercion to integer range
8: In CPL_read_gdal(as.character(x), as.character(options), as.character(driver), :
NAs introduced by coercion to integer range
9: In CPL_read_gdal(as.character(x), as.character(options), as.character(driver), :
NAs introduced by coercion to integer range

I don't know what is going on, can you give me some advice?
Thank you very much!

installation error

Error: Failed to install 'biodivMapR' from GitHub:
(converted from warning) cannot remove prior installation of package ‘matrixStats’

Error during installing the package

Hi. I have RStudio version 3.5.2 installed in my system. I have downloaded the .zip file of the package and trying to install it using the command:
<install.packages('C:\Users\asd\Downloads\biodivMapR-master.zip')>

After running the above command I am getting the following warning:

"Installing package into ‘C:/Users/asd/Documents/R/win-library/3.5’
(as ‘lib’ is unspecified)
Warning in install.packages :
package ‘C:\Users\asd\Downloads\biodivMapR-master.zip’ is not available (for R version 3.5.2)"

Post this when I am trying to import the library(biodivMapR) I am getting the following:
"Error in library(biodivMapR) : ‘biodivMapR’ is not a valid installed package"

Problem with Component selection for dimensionality reduction

Not related to the code itself, but how to select the principle components in a correct manner?

I was following the tutorial and after obtaining the images of 8 bands generated by PCA, I find it confusing to figure out which bands should be discarded and which bands should be chosen and written in "Selected_Components.txt".

I used QGIS to render the image of each band so the images in my end are not using the same style as the results showed in your tutorial website. But I am not sure if the style matters here in order to choose the desired principle components.

Edit: Thanks in advance. But if this thread were not appropriate to be post here since it is not relevant to the code, feel free to delete it.

Diversity maps output resolution

Hi dear developers

Is it possible to modify the output resolution of the resulting alpha and beta diversity maps made from a Sentinel 2A image? Even when I have modified the "window_size" parameter, the output pixel size is 100m. This is the default output resolution?

Regards!

JP.

the NAN values exist, it doesn't work

when I performed the model biodivmapR, it did not work, because the NAN values exist. How could I do for this.

What is the R version to run dissUtils?