jonesor/compadredb

Code and data for the COMPADRE Plant Matrix Database and COMADRE Animal Matrix Database

compadredb's Issues

Add db type ('COMADRE' or 'COMPADRE') to version slot?

Suggest adding db type to version info when the db is initially compiled, so that Rcompadre methods (e.g. show and fetchDB) know for sure which database it is and can print messages accordingly, without having to resort to 'tricks' like looking at db$Kingdom or other columns that may be removed by the user.

Could combine with the $Version element, e.g. "COMPADRE 4.0.1", but I think I'd prefer a separate element e.g.

$Database
$Version
$DateCreated
$Agreement

Improve function documentation for collapse matrix functions

Issue by tdjames1
Thursday Jun 22, 2017 at 13:34 GMT # Sample: Friday Sep 13, 2013 at 22:58 GMT
Originally opened as tdjames1#6

---

E.g. use descriptions of parameters matU, matF etc from lifetimeRepEvents() documentation.

Failed script Plot_LifeCycle_Diagram.R

I'm trying these scripts but can't make this one work.
After I execute the last command I get a warning:

Warning message:
In sample.int(x, size, replace, prob) :
  '.Random.seed' is not an integer vector but of type 'NULL', so ignored

And it opens a browser window with this stack:

abort(0) at jsStackTrace@file:///tmp/Rtmp8kUDC7/viewhtml17e655528a/lib/viz-0.3/viz.js:5:22063
stackTrace@file:///tmp/Rtmp8kUDC7/viewhtml17e655528a/lib/viz-0.3/viz.js:5:22246
abort@file:///tmp/Rtmp8kUDC7/viewhtml17e655528a/lib/viz-0.3/viz.js:28:10646
nullFunc_iii@file:///tmp/Rtmp8kUDC7/viewhtml17e655528a/lib/viz-0.3/viz.js:5:662060
a8@file:///tmp/Rtmp8kUDC7/viewhtml17e655528a/lib/viz-0.3/viz.js:21:31632
iC@file:///tmp/Rtmp8kUDC7/viewhtml17e655528a/lib/viz-0.3/viz.js:9:83337
aD@file:///tmp/Rtmp8kUDC7/viewhtml17e655528a/lib/viz-0.3/viz.js:9:102096
uF@file:///tmp/Rtmp8kUDC7/viewhtml17e655528a/lib/viz-0.3/viz.js:9:173803
pG@file:///tmp/Rtmp8kUDC7/viewhtml17e655528a/lib/viz-0.3/viz.js:9:204482
xc@file:///tmp/Rtmp8kUDC7/viewhtml17e655528a/lib/viz-0.3/viz.js:11:738
@file:///tmp/Rtmp8kUDC7/viewhtml17e655528a/lib/viz-0.3/viz.js:28:372
ccallFunc@file:///tmp/Rtmp8kUDC7/viewhtml17e655528a/lib/viz-0.3/viz.js:5:8392
Viz@file:///tmp/Rtmp8kUDC7/viewhtml17e655528a/lib/viz-0.3/viz.js:47:25
.renderValue@file:///tmp/Rtmp8kUDC7/viewhtml17e655528a/lib/grViz-binding-0.9.0/grViz.js:38:24
window.HTMLWidgets.staticRender/</<@file:///tmp/Rtmp8kUDC7/viewhtml17e655528a/lib/htmlwidgets-0.8/htmlwidgets.js:625:11
forEach@file:///tmp/Rtmp8kUDC7/viewhtml17e655528a/lib/htmlwidgets-0.8/htmlwidgets.js:55:7
window.HTMLWidgets.staticRender/<@file:///tmp/Rtmp8kUDC7/viewhtml17e655528a/lib/htmlwidgets-0.8/htmlwidgets.js:551:7
forEach@file:///tmp/Rtmp8kUDC7/viewhtml17e655528a/lib/htmlwidgets-0.8/htmlwidgets.js:55:7
window.HTMLWidgets.staticRender@file:///tmp/Rtmp8kUDC7/viewhtml17e655528a/lib/htmlwidgets-0.8/htmlwidgets.js:549:5
@file:///tmp/Rtmp8kUDC7/viewhtml17e655528a/lib/htmlwidgets-0.8/htmlwidgets.js:638:7

Anyne has any idea what that might mean?

Fix example in extractVitalRates()

Issue by tdjames1
Thursday Jun 22, 2017 at 13:14 GMT # Sample: Friday Sep 13, 2013 at 22:58 GMT
Originally opened as tdjames1#2

---

The structure of the matrix given in the example is not appropriate for the vital rate extraction procedure.

Create example for rearrangeMatrix() where matrix contains non-reproductive stages

Issue by tdjames1
Thursday Jun 22, 2017 at 13:20 GMT # Sample: Friday Sep 13, 2013 at 22:58 GMT
Originally opened as tdjames1#3

---

Create an example for rearrangeMatrix() that demonstrates the function's purpose, i.e. a matrix containing non-reproductive stages.

rearrangeMatrix() fails when matF contains NA

Issue by tdjames1
Wednesday Jun 28, 2017 at 16:07 GMT # Sample: Friday Sep 13, 2013 at 22:58 GMT
Originally opened as tdjames1#7

---

Example from Bernt-Erik Sæther, Øvind Bakke. 2000. Life history variation and contribution of different demographic traits to the population growth rate in birds: a comparative approach. Ecology 81: 642-653.

 saetherMatrices$`Diomedea exulans`
      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11]
 [1,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00  0.00    NA
 [2,] 0.92 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00  0.00  0.00
 [3,] 0.00 0.92 0.00 0.00 0.00 0.00 0.00 0.00 0.00  0.00  0.00
 [4,] 0.00 0.00 0.92 0.00 0.00 0.00 0.00 0.00 0.00  0.00  0.00
 [5,] 0.00 0.00 0.00 0.92 0.00 0.00 0.00 0.00 0.00  0.00  0.00
 [6,] 0.00 0.00 0.00 0.00 0.92 0.00 0.00 0.00 0.00  0.00  0.00
 [7,] 0.00 0.00 0.00 0.00 0.00 0.92 0.00 0.00 0.00  0.00  0.00
 [8,] 0.00 0.00 0.00 0.00 0.00 0.00 0.92 0.00 0.00  0.00  0.00
 [9,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.92 0.00  0.00  0.00
[10,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.92  0.00  0.00
[11,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00  0.92  0.92

Decomposed using values above the diagonal as the fecundity matrix, remaining values as growth/transition.

[1] "Species:  Diomedea exulans"
      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11]
 [1,]    0    0    0    0    0    0    0    0    0     0    NA
 [2,]    0    0    0    0    0    0    0    0    0     0     0
 [3,]    0    0    0    0    0    0    0    0    0     0     0
 [4,]    0    0    0    0    0    0    0    0    0     0     0
 [5,]    0    0    0    0    0    0    0    0    0     0     0
 [6,]    0    0    0    0    0    0    0    0    0     0     0
 [7,]    0    0    0    0    0    0    0    0    0     0     0
 [8,]    0    0    0    0    0    0    0    0    0     0     0
 [9,]    0    0    0    0    0    0    0    0    0     0     0
[10,]    0    0    0    0    0    0    0    0    0     0     0
[11,]    0    0    0    0    0    0    0    0    0     0     0
      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11]
 [1,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00  0.00  0.00
 [2,] 0.92 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00  0.00  0.00
 [3,] 0.00 0.92 0.00 0.00 0.00 0.00 0.00 0.00 0.00  0.00  0.00
 [4,] 0.00 0.00 0.92 0.00 0.00 0.00 0.00 0.00 0.00  0.00  0.00
 [5,] 0.00 0.00 0.00 0.92 0.00 0.00 0.00 0.00 0.00  0.00  0.00
 [6,] 0.00 0.00 0.00 0.00 0.92 0.00 0.00 0.00 0.00  0.00  0.00
 [7,] 0.00 0.00 0.00 0.00 0.00 0.92 0.00 0.00 0.00  0.00  0.00
 [8,] 0.00 0.00 0.00 0.00 0.00 0.00 0.92 0.00 0.00  0.00  0.00
 [9,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.92 0.00  0.00  0.00
[10,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.92  0.00  0.00
[11,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00  0.92  0.92
Error in Rep[1]:Rep[length(Rep)] : argument of length 0

Standardize column types (character vs. factor)?

The current X.X.X versions of COM(P)ADRE use a mix of factor and character for the character-type columns. Suggest standardizing. In terms of Rcompadre, I have a slight preference for character because that's the default for tibbles.

SpeciesAuthor          "character"
SpeciesAccepted        "character"
CommonName             "character"
Genus                  "character"
Family                 "factor"   
Order                  "factor"   
Class                  "factor"   
Phylum                 "factor"   
Kingdom                "factor"   
OrganismType           "factor"   
DicotMonoc             "factor"   
AngioGymno             "factor"   
Authors                "character"
Journal                "character"
YearPublication        "character"
DOI.ISBN               "character"
AdditionalSource       "character"
StudyDuration          "numeric"  
StudyStart             "numeric"  
StudyEnd               "numeric"  
AnnualPeriodicity      "character"
NumberPopulations      "numeric"  
MatrixCriteriaSize     "factor"   
MatrixCriteriaOntogeny "factor"   
MatrixCriteriaAge      "factor"   
MatrixPopulation       "character"
Lat                    "numeric"  
Lon                    "numeric"  
Altitude               "numeric"  
Country                "factor"   
Continent              "factor"   
Ecoregion              "character"
StudiedSex             "factor"   
MatrixComposite        "factor"   
MatrixTreatment        "character"
MatrixCaptivity        "factor"   
MatrixStartYear        "numeric"  
MatrixStartSeason      "factor"   
MatrixStartMonth       "numeric"  
MatrixEndYear          "numeric"  
MatrixEndSeason        "factor"   
MatrixEndMonth         "numeric"  
MatrixSplit            "factor"   
MatrixFec              "factor"   
Observation            "factor"   
MatrixDimension        "numeric"  
SurvivalIssue          "numeric"  

Ternary plot worksheet/vignette is broken - fix it!

The ternary plot function in the vignette (Workshops folder) is broken.
Could you take a look @robcito

plotLifeCycle order

I think that the stages in the function plotLifeCycle get automatically re-ordered to alphabetical, unless they are given a number before them (which is not very sensible). This makes the reading of the lifecycle from left-to-right a bit less straight forward. It'd be great if we could let the authors choose the order in which the stages appear

mage: move fxn level documentation into .R files

on call with @robcito and @tdjames1 recently, we talked about moving function documentation into the .R files themselves - just a note here to work on that

Possible normalized database structure for com(p)adreDB

Thanks for the chat this morning @jonesor and @robcito. Based on our
conversation I did a little thinking about a complementary data structure to current
R specific one that is more database/language agnostic. Here are some
initial thoughts on what a normalized database structure might look like for this
data. It would require processing to make it as usable as the current structure
is within R, but would allow a broader array of computational tools to access
and use the data.

The basic structure would be three tables, one each to hold the data in
metadata, mat, and matrixClass. metadata is already a table so it could
remain as it is, but with a new StudyID column corresponding to the current
position within the main R object for linking to other tables. It could also be
further normalized by splitting it up based on type of information (e.g.,
splitting out a separate Taxonomy table to avoid some duplication).

mat would have one row for each cell occurring in a matrix and store the
StudyID, the Matrix letter, row, column, and value. E.g.,:

StudyID,Mat,Row,Col,Value
1,A,1,1,0.1
1,A,1,2,0.3
1,A,2,1,0.2
1,A,2,2,0.2
1,U,1,1,0.1
1,U,1,2,0.3
....
7024,C,10,10,0

matrixClass would basically be a combination of the 7024 matrixClass objects
into one large table. E.g.,:

StudyID,MatrixClassNumber,MatrixClassOrganized,MatrixClassAuthor
1,1,active,Seedling
1,2,active,Small reproductive
1,3,active,Large reproductive
...
7024,10,active,Largest adult(> 5 m)

I'm still just getting a feel for everything going on in the data so I'm sure
I've missed some things and I didn't fully process this mornings conversation around
the vectorized versions of this data so apologies if I'm just restating something you've
already thought of.

Resolve various queries in reprodStages()

Issue by tdjames1
Thursday Jun 22, 2017 at 13:28 GMT # Sample: Friday Sep 13, 2013 at 22:58 GMT
Originally opened as tdjames1#4

---

There are various unresolved queries in this function relating to the way it should work. Summarising these:

• What is the best form for the output?
• How should we handle dormant stages?
• Is there any justification for stages to be combined/collapsed e.g. propagule and pre-reproductive or reproductive and post-reproductive?

More details are in the function documentation:
https://github.com/tdjames1/compadreDB/blob/master/Mage/R/reprodStages.R

SpeciesAccepted inconsistencies

In compadre, SpeciesAccepted is separated by a space, i.e. "Genus species". In comadre the separator is an underscore, i.e. "Genus_species". Something to fix to ensure consistency between the two databases.

Unresolved merge conflicts in kentropy.R and dentropy.R

There are unresolved merge conflicts where documentation has been added/function signature has been modified for these two files. This causes install_github() to fail.

qn: raw data archive integration

Quick question that came up in the LivingNorway.no workshop – in terms of database integration: as far as I can tell, the only link to the original data in COMADRE/COMPADRE is the citation (typically to a paper). Is there any other way to see if the original data (i.e. not the processed matrices, but the data underlying them) is also archived somewhere?

Decide what to do about standardizedVitalRates()

Issue by tdjames1
Thursday Jun 22, 2017 at 13:32 GMT # Sample: Friday Sep 13, 2013 at 22:58 GMT
Originally opened as tdjames1#5

---

This function is supposed to handle the whole process of collapsing a matrix to standard form and calculating standardized vital rates. Need to figure out if it does what it's supposed to do.

https://github.com/tdjames1/compadreDB/blob/master/Mage/R/standardizedvitalrates.R

mage: write unit tests

on call with @robcito and @tdjames1 recently, we talked about adding unit testing - just a note here to work on tests

mage: maybe break into two packages

on call with @robcito and @tdjames1 recently, we talked about possibly splitting Mage into two packages,

• one focusing on interacting with compadre/comadre and other databases
• one focusing on general purpose set of functions for working with the data above

thoughts, feeling, etc.

user guide variable descriptions do not match compadre$metadata columns

compadre$metadata column names do not match the descriptions in the user guide pdf. Some are present in the user guide but not in compadre$metadata (e.g., degree, minute, second columns for both latitude and longitude are described in the guide whereas the .Rdata object holds single columns for both Lat and Lon in units of decimal degrees). In contrast, other column headers appear in compadre$metadata, but are not described in the user guide (e.g., CommonName).

possible option with dplyr for subsetting

# load compadre, then:
library("dplyr")

# function
subsetdb2 <- function(x, ...) {
  x$metadata %>% filter(...)
}

# Class == "Pinopsida"
subsetdb2(compadre, Class == "Pinopsida")

# Class == "Pinopsida" and TaxonomicStatus == "Synonym"
subsetdb2(compadre, Class == "Pinopsida", TaxonomicStatus == "Synonym")

Negative elasticity from matrixElementPerturbation function

x$mat[[62]]
$matA
         A1    A2     A3    A4    A5    A6     A7
[1,] 0.0000 0.000 0.0000 0.000 0.000 0.000 705.75
[2,] 0.6218 0.000 0.0000 0.000 0.000 0.000   0.00
[3,] 0.0000 0.598 0.0000 0.000 0.000 0.000   0.00
[4,] 0.0000 0.000 0.5666 0.000 0.000 0.000   0.00
[5,] 0.0000 0.000 0.0000 0.488 0.000 0.000   0.00
[6,] 0.0000 0.000 0.0000 0.000 0.217 0.000   0.00
[7,] 0.0000 0.000 0.0000 0.000 0.000 0.725   0.00

$matU
         U1    U2     U3    U4    U5    U6 U7
[1,] 0.0000 0.000 0.0000 0.000 0.000 0.000  0
[2,] 0.6218 0.000 0.0000 0.000 0.000 0.000  0
[3,] 0.0000 0.598 0.0000 0.000 0.000 0.000  0
[4,] 0.0000 0.000 0.5666 0.000 0.000 0.000  0
[5,] 0.0000 0.000 0.0000 0.488 0.000 0.000  0
[6,] 0.0000 0.000 0.0000 0.000 0.217 0.000  0
[7,] 0.0000 0.000 0.0000 0.000 0.000 0.725  0

$matF
     F1 F2 F3 F4 F5 F6     F7
[1,]  0  0  0  0  0  0 705.75
[2,]  0  0  0  0  0  0   0.00
[3,]  0  0  0  0  0  0   0.00
[4,]  0  0  0  0  0  0   0.00
[5,]  0  0  0  0  0  0   0.00
[6,]  0  0  0  0  0  0   0.00
[7,]  0  0  0  0  0  0   0.00

$matC
     C1 C2 C3 C4 C5 C6 C7
[1,]  0  0  0  0  0  0  0
[2,]  0  0  0  0  0  0  0
[3,]  0  0  0  0  0  0  0
[4,]  0  0  0  0  0  0  0
[5,]  0  0  0  0  0  0  0
[6,]  0  0  0  0  0  0  0
[7,]  0  0  0  0  0  0  0

> matrixElementPerturbation(matU = x$mat[[62]]$matU, matF = x$mat[[62]]$matF,matC = x$mat[[62]]$matC)[6:10]
  EStasis EProgression ERetrogression EFecundity EClonality
1       0    -1894.731              0  0.1428571          0

Doesn't seem right to me