ropensci/rvertnet

Wrapper to the VertNet API

rvertnet

rvertnet is a client for interacting with VertNet.org.

VertNet.org API docs: https://github.com/VertNet/webapp/wiki/The-API-search-function

Installation

Stable CRAN version

install.packages("rvertnet")

Development version from GitHub

remotes::install_github("ropensci/rvertnet", build_vignettes = TRUE)

Get started

View vignette on ropensci website

Or, view vignette locally in your web browser

browseVignettes("rvertnet")

Or, view vignette in the Rstudio help pane

vignette("rvertnet")

Contributors

• Dave Slager @slager
• Scott Chamberlain @sckott
• Chris Ray @Pika8tona
• Vijay Barve @vijaybarve

Meta

• Please report any issues or bugs.
• License: MIT
• Get citation information for rvertnet in R doing citation(package = 'rvertnet')
• Please note that this package is released with a Contributor Code of Conduct. By contributing to this project, you agree to abide by its terms.

rvertnet's Issues

remove binomial from example

To keep folks from accidentally limiting their search results by entering a binomial instead of a specific epithet, these lines can be omitted from the headers in searchbyterm and bigsearch:

' # Find records in multiple locations

' out <- searchbyterm(specificepithet = "mustela nigripes")

'

Looks like they were a holdover that I missed--sorry.

bigsearch uses wrong mappable query

If I run the following code:
bigsearch(genus = 'Crotalus', mappable = TRUE, rfile = 'Crotalus_vertnet', email = '[email protected]')

... I quickly get an email with the resulting file, which has zero records.
But in the email, the query reads: "Query: genus:Crotalus mappable:true"

I believe it should be "mappable:1".
I haven't checked to see if this is happening for all logical flags.

DESCRIPTION should include Url and BugReports

Pointing to this repo

Cursor

@Pika8tona Do you know if you have used the cursor functionality in rvertnet? I don't see it anywhere in the package, but maybe you have used it.

Or maybe bigsearch() is a replacement for paging over results to get more than 1000?

cc @dbloom any thoughts?

Fix to allow certain parameters to handle multiple values

e.g, year

Need one or two functions to do basic visualization of data returned in rvertnet

the param specificepithet shouldn't represent the binomial/trinomial species name

http://rs.tdwg.org/dwc/terms/#specificEpithet - so change to species - in one or more fxns, can't remember which ones

put diff. email address in egs so i'm not pinged when they're run

dumby email or possibly my email with +rvertnet-egs

Change searchbyterm() to allow full text query

will make it so that we can use searchbyterm() in spocc, and make it easier to get what we want in that pkg

httr::content parse manually and encoding to utf8

bigsearch should fail better

• when given no inputs except email and file name, should fail (or does this request a dump of all of vertnet? if so, I think we'd want that to fail, no?)
• rfile and email are required fields, should fail if not given

Help users read in data from bigsearch request

a user reports errors on trying to read in tsv file from a request to bigsearch()

fread seems to work fine so far, suggest that or possibly use as import and have a wrapper function for it?

Change mappable values to TRUE/FALSE instead of 1/0

more R like style

Data dumps

http://blog.vertnet.org/ see post "The (Data)ONE thing about VertNet and Big Data"

Check that old version problems don't still exist

from use email:

vertoccurrence(grp='herp', t='Crotalus oreganus')
Error in textConnection(text, encoding = "UTF-8") :
invalid 'text' argument

z <- vertoccurrence(grp='fish', t='Ventrifossa macropogon')

In this instance, for whatever reason, the longitude and latitude columns are of class factor. This means that if you simply plot the occurrences, the coordinates are coerced directly from factor to numeric, resulting in nonsensical coordinates. Seems to me that perhaps these columns should at least be coerced to character prior to being returned by the function.

Add ... for curl options to all fxns

Add skip_on_cran()

binomial "species" vs specific epithet

In searchbyterm and bigsearch the user should be prompted for just the specific epithet ("leo" in "Panthera leo") when entering "species", because the "specificepithet" field in VertNet usually does not contain a binomial. For example, when I search for sp="ochotona princeps" I get 78 results, compared to over 6800 results for genus="ochotona" combined with sp="princeps"

If no args passed to fxns, make them fail

as I don't think there is a use case for no args passed to any fxns

readme fixes

These packages contain README.md files with invalid HTML output created
by pandoc 1.12.4.2 according to W3C-validator.

I attach the HTML errors and warnings found below, and will put copies
of the corresponding HTML files up at
http://www.r-project.org/nosvn/pandoc.

Please investigate the problems and fix as needed.

Afaics, many of the problems are caused by adding "raw" HTML elements in
the README.md files and not realizing that the default output format
"html" is XHTML 1 (and not HTML 5). E.g., a raw
results in an

end tag for "br" omitted, but OMITTAG NO was specified

error.

Best
-k

rvertnet.html:
  Valid: FALSE (errors: 1, warnings: 0)
  Errors:
    line  col  message
      30  109  required attribute "alt" not specified

ggplot2 changes

2 tiny things in vignette

Very small things in the nice vignette: Will "[email protected]" in the vignette be confusing, since it's followed in the fake output by "[email protected]"? Also "latitud" missing an e appears in the vignette.

Vertnet API

There seems to be a new web interface (http://portal.vertnet.org/) for Vernet - but I don't see anything about a new API. Anybody know what's going on with Vetnet's API?

Most of the old function call examples work, but it seems odd that there is no way to get data on mammals given that its vertebrate data, especially given that you can get mammal data on their new web interface.

Where is bigsearch function

hi @Pika8tona - I don't see the bigsearch() function anywhere. Is it in your local branch on your computer, and didn't make it up here somehow? There is a .Rd file for it, but no function definition for it.

Bug in stateprovince param

When using boolean, requires that parentheses go around the entire string, but apparently this doesn't apply to other params

Note the use of parentheses to group together the two possible values for the "stateprovince" field.

Remove some files

@Pika8tona

Some files shouldn't be in the root directory of the package:

• vertnet-manual.pdf
• rvertnet_0.2.tar.gz

The manual will be created on build steps on CRAN when it gets to cran, but you can keep it therer for now.

the tarball isn't needed as when people install from Github they can do install_github("ropensci/rvertnet") and its not needed there, and when its on cran, they create the appropriate tarballs for us.

Add these two files to .Rbuildignore so that when we run check they aren't checked

externalptr error with CRAN version 0.2.2

I installed rvertnet from CRAN by habit rather than via github, so it installed version 0.2.2. This resulted in the documentation/vignettes online not matching, old functions, and a couple errors with functions. This might be irrelevant given the github version but it might be worth submitting a newer version to CRAN (or getting rid of the the CRAN version and just using GitHub). Feel free to close without comment if this is not useful.

install.packages("rvertnet")

Installing package into ‘/Users/Dan/Library/R/3.1/library’
(as ‘lib’ is unspecified)

  There is a binary version available (and will be installed) but the
  source version is later:
         binary source
rvertnet  0.2.2  0.4.1

library(rvertnet)
library(XML)

Error using vertoccurrence

>nvir <- vertoccurrence(t="Notophthalmus viridescens", grp="herp")
Error in textConnection(text, encoding = "UTF-8") : 
  invalid 'text' argument

Error using vertsearch:

> nvir <- vertsearch(taxon="Notophthalmus viridescens")
Error in vertsearch(taxon = "Notophthalmus viridescens") : 
  unused argument (taxon = "Notophthalmus viridescens")

> nvir <- vertsearch(sp="Notophthalmus viridescens")
Error in out$records : object of type 'externalptr' is not subsettable

Function bigsearch not available:

> ?bigsearch
No documentation for ‘bigsearch’ in specified packages and libraries:
you could try ‘??bigsearch’

dumps: new trait dataset, and possibly new URLs?

http://datacommons.cyverse.org/browse/iplant/home/shared/commons_repo/curated/VertNet_Traits

http://blog.vertnet.org/post/151662714981/new-vertnet-taxon-based-snapshots-are-now

Work on revamping fxns with new API

https://github.com/VertNet/webapp/wiki/The-API-search-function

Missing else

@Pika8tona Curious what you meant to put here (see the arrow and ? below), this is in the vertwrapper() function: should that last else be removed?

  if(fxn == "vertsearch"){
    qstr <- paste(args, collapse = "%20")
  } else if(fxn == "searchbyterm" | fxn == "bigsearch"){
    qstr <- paste(names(args), args, sep = ":", collapse = "%20")
    qstr <- gsub(":>", ">", qstr <- gsub(":<", "<", qstr <- gsub(":=", "=", qstr))) # Numerical query
  } else if(fxn == "spatialsearch"){
    qstr <- paste("distance(location,geopoint(", 
                  paste(args$lat, args$long, sep = ","), "))<", args$radius, sep = "")
  } else { # Additional functions
    ##### ????? <<<<------
  } 

update docs that >1000 works now - cursor implemented

occurrenceid changed to occurrenceID

id upcased,

causing problems i think

Search by occurrence ID update

VertNet/webapp#531 (comment)

See if any changes needed here, and in spocc

Make basic vignette

👍 🆘

Write small test suite

CRAN check notes

checking dependencies in R code ... NOTE
Namespaces in Imports field not imported from:
  ‘RCurl’ ‘RJSONIO’
  All declared Imports should be used.
See the information on DESCRIPTION files in the chapter ‘Creating R
packages’ of the ‘Writing R Extensions’ manual.

checking R code for possible problems ... NOTE
vertmap: no visible binding for global variable ‘long’
vertmap: no visible binding for global variable ‘lat’
vertmap: no visible binding for global variable ‘group’
vertmap: no visible binding for global variable ‘Longitude’
vertmap: no visible binding for global variable ‘Latitude’

checking Rd line widths ... NOTE
Rd file 'vertlocations.Rd':
  \examples lines wider than 100 characters:
     vertlocations(p="POLYGON((-93.998292265615 32.615318339629,-92.471192656236 32.606063985828,-92.635987578112 31.235349580893,-90.988038 ... [TRUNCATED]

Rd file 'vertoccurrence.Rd':
  \examples lines wider than 100 characters:
     vertoccurrence(p="POLYGON((-93.998292265615 32.615318339629,-92.471192656236 32.606063985828,-92.635987578112 31.235349580893,-90.98803 ... [TRUNCATED]
     vertoccurrence(t="notropis",num=100, cols="Latitude,Longitude,Family,ScientificName,IndividualCount,Remarks")

Rd file 'vertoccurrencecount.Rd':
  \examples lines wider than 100 characters:
     vertoccurrencecount(p="POLYGON((-93.998292265615 32.615318339629,-92.471192656236 32.606063985828,-92.635987578112 31.235349580893,-90. ... [TRUNCATED]

Rd file 'vertproviders.Rd':
  \examples lines wider than 100 characters:
     vertproviders(p="POLYGON((-93.998292265615 32.615318339629,-92.471192656236 32.606063985828,-92.635987578112 31.235349580893,-90.988038 ... [TRUNCATED]

Rd file 'verttaxa.Rd':
  \examples lines wider than 100 characters:
     verttaxa(p="POLYGON((-93.998292265615 32.615318339629,-92.471192656236 32.606063985828,-92.635987578112 31.235349580893,-90.98803835936 ... [TRUNCATED]

These lines will be truncated in the PDF manual.

Shit is broken

Calls don't work, figure out what's wrong

use dplyr::bind_rows instead of dplyr::rbind_all

Return metadata in data requests

incorporate trait data into package

docs: https://github.com/VertNet/webapp/wiki/The-API-search-function#trait

---

e.g.'s

http://portal.vertnet.org/search?q=genus:Ochotona+hasmass:1+haslength:1

generates a list of all pikas that have mass and length measurements.

http://portal.vertnet.org/search?q=genus:Ochotona+lengthinmm+%3E=+100+lengthinmm+%3C=+1000+lengthtype:total+length+hasmass:1+haslength:1

generates the same list with those specimens between 100mm and 1000mm in length.

Mark old functions as defunct

Any changes?

Any changes we need to make given changes noted in http://blog.vertnet.org/post/131026116546/vertnet-portal-now-with-new-features-and

Import all non-base R fxns

see https://cran.rstudio.com/web/checks/check_results_rvertnet.html

Figure out API key thing for vertnet

We currently use an API key that is provided in the function itself. Vijay, can you please figure out if they are going to provide API keys to users, or if we should just always use this one key in to the future.

Package does not load with install_github

Fix this...

Vignette update

Update vignette with:

• Cursor use, via limit parameter
• other ?

use tibble

https://cran.rstudio.com/web/packages/tibble/

Pretty much ready to go to CRAN

@dbloom @Pika8tona Any final thoughts before we send this to CRAN?

readme xhtml

From kurt hornik

These have README.md files which when converted to (X)HTML using a
current version of pandoc show problems when validated using W3C Markup
Validator, see below.

Most of these problems are caused by using images without giving a name
(so the required alt attribute for <img> is not provided), or using <br>
instead of <br/>.

Pls fix these problems in your README.md files for your next release: in
all cases I inspected, the fixes were obvious and confirmation using
pandoc and W3C markup validator seemed unnecessary.

Please also visit your package check web page at http://cran.r-project.org/web/checks/check_results_PACKAGENAME.html to see if other problems need to be addressed as well.

Namespace base R pkg fxn calls, drop any not needed deps