nwfsc-math-bio/vrdata

Spawner and fraction wild data for the NWFSC Status Reviews.

README

See the WEBSITE for the data information. This Readme is for the package and data maintainers.

New data

• Add new csv files to the data-raw folder.
• See the wiki for instructions on data format.
• Push to GitHub and the data will be automatically processed for the website via a GitHub Action.

The GitHub Action

The following will be run automatically.

Changing the man files and vignettes

The man files appear under the Reference tab and the vignettes appear under the Articles tab. The R and Rmd files for these are created by the function process_data_raw(). When you type

process_data_raw()

It processes the csv file in data-raw folder into a .rda object in data, a .R file, and a .Rmd vignette using the Roxygen header on the csv file. Change this file to change the look of the man or article files.

When the package is built, you can do ? to get the meta information.

Updating the webpage locally

Note, you can run process_data_raw() locally, but the GitHub repo is set up with a GitHub Action to automatically run process_data_raw() whenever a push to the data-raw folder happens.

Manual workflow

• Build the package (click 'Install and Restart' in the build tab)
• Add csv file to data-raw
• Run process_data_raw()
• Build the documentation, then build package
• Run pkgdown::build_site()

library(VRData)
process_data_raw()
devtools::document(roclets = c('rd', 'collate', 'namespace'))
pkgdown::build_site()

Disclaimer

This repository is a scientific product and is not official communication of the National Oceanic and Atmospheric Administration, or the United States Department of Commerce. All NOAA GitHub project code is provided on an 'as is' basis and the user assumes responsibility for its use. Any claims against the Department of Commerce or Department of Commerce bureaus stemming from the use of this GitHub project will be governed by all applicable Federal law. Any reference to specific commercial products, processes, or services by service mark, trademark, manufacturer, or otherwise, does not constitute or imply their endorsement, recommendation or favoring by the Department of Commerce. The Department of Commerce seal and logo, or the seal and logo of a DOC bureau, shall not be used in any manner to imply endorsement of any commercial product or activity by DOC or the United States Government.

vrdata's Issues

v2020 Project Backlog

2020 Data Visualization

Release 2.3

• #12
• #21

Release 2.4

• #22
• #35

Release 2.X

Need to decide if this is part of VRData or not

• E5 Include summary table with output
• E6 Create a basic shiny app (dashboard) that visualizes bigtable [optional]
• E7 Google Drive integration

S1.7 Misc clean-up

• Add Viability Report citation
• Add download to Rd files
• Update installation to release not main
• Clean up the pkgdown site when rebuilding. Keeps just adding on stuff. Add to GH Action. Don't use pkgdown::clean_site(pkg = ".") because I need data dir and can't create wo pkgdown complaining.

S1.3 Create branch for code changes and then branch off that for 2020 data

• Create structure
• Add 2020 data to 2020data branch
• Do the data clean up
• Clean up the branch

S1.4 Make sure VRData doesn't choke if there is no header

• Create the 2020data branch with current VRData code.
• Build the pkgdown site
• Check for problems
• If problems, fix code in code branch. Pull into 2020data branch.
• Check that code branch still builds pkgdown aok.

Add review for new data files

Figure how to have changes post to a branch or something for review before merging.

research google drive integration

• Contact Emily re example.

Add Mari, Katie and Mindy as authors

Data compilers to package

S3 clean up inst/extdata

Not sure what the files in there are. Don't think they are needed.

Task: Complete data documentation QC for 2020 VR

Who: Data team + Eli

• All contributor and citation entries complete
• All source notes entered
• Archive raw data files and csv files
• Data entered into SPS

S3 Add ReadMe and move current Readme for developers elsewhere

Probably want the index.md and readme to be the same

S3 Add DOI for VRData via Zenodo

After has been pushed up review. DOI

S3 Check data Rd files for full meta-data

Compiled by: instead of author

Contributor:
WA Department of F & W

Citation:
available at https://fortress.wa.gov/dfw/score/score/; http://odfwrecoverytracker.org/metadata/; https://www.monitoringmethods.org

East Fork South Fork Salmon River summer Chinook, Secesh River summer Chinook, and Snake River Lower Mainstem fall Chinook Natural Origin Spawner Abundance Dataset (1949-2018). Spawner abundance data.

Nez Perce Tribe. Protocol and methods available at https://www.cbfish.org/Document.mvc/Viewer/P165414; https://www.cbfish.org/Document.mvc/Viewer/P164600. Personal communication with Mari Williams, NOAAF NWFSC/OAI Dec 2019.

Description (what is this a data set of)
contributor,
Citation
methods,
download date
data source(s)
compiler,

Format add info if number
ESU/DPS (constrained field)
Fracwild (decimal number between 0 and 1).
Year. Integer. See details for ESU/DPS comments.
NUMBER_OF_SPAWNERS Total spawners or escapement.

Reports

spawners on same plot
y-scaling

E3 Clean up the citation, DOI, and documentation

• #29
• #33
• #28
• #11
• #1
• #34
• #24

Code googledrive integration

Using https://github.com/RVerse-Tutorials/GoogleDrive1 example, code a script to

• Check Google Drive folder for changes
• Download the changed files and add to data folder
• Commit to GitHub

S3 Add common metrics to vignettes

E4 Final data check

• #27

Add source and citation info for Puget Sound

• Notes from Mindy (everyone)
• Update in SPS data file (Mari)
• Update csv files from SPS data (Eli)
• Double-check that everything is updated and no blank entries

S1.6 Check that none of the vignette code or Rd code is using columns that don't exist

here's what

Write basic data workflow document

E2 Improve usability of the data documentation

• S2.1 Add download raw data button to the data man files
• S2.3 Clean-up the Rd output

Fix the download names for raw data

Right now super long

S4 Pull in final version of SPS data

Items to fix

• Line 6312-6515 is all blank for spawners, fracwild and catch. Probably good to put -99 there to make clear that these are years with missing data not a mistake.

• Same with lines 5778-5792

• The FRACWILD column still has some blanks (besides those above). That has tripped us up in the past since these are interpreted as -99. Might be good to make them -99 to be clear that they are.

• The ESU column has 3 ESUs that don't follow the format of all the others: Hood Canal, Ozette Lake Sockeye Salmon ESU, Puget Sound. All the other ESUs look something like "Steelhead (Puget Sound DPS)"

• The RUN_TIMING column has a variety of formats. e.g. Su, Summer, and SumR. Wi, Winter and WR.

• NUMBER_OF_SPAWNERS column has blanks. Those should be -99 to be clear that they are missing.

• SPECIES has both Chinook and Chinook salmon

• Are we using BROOD_YEAR as the year column? I thought we switched to YEAR.

• Are we using START and END columns? Mostly they are blank. If we are, note that some of the END values are different, e.g. 1968-2019 instead of 2019.

Add Mari's notes re citation info

See issue in SR2020 repo

create template for data file meta data

Compiled by: instead of author

Contributor:
WA Department of F & W

Citation:
available at https://fortress.wa.gov/dfw/score/score/; http://odfwrecoverytracker.org/metadata/; https://www.monitoringmethods.org

East Fork South Fork Salmon River summer Chinook, Secesh River summer Chinook, and Snake River Lower Mainstem fall Chinook Natural Origin Spawner Abundance Dataset (1949-2018). Spawner abundance data.

Nez Perce Tribe. Protocol and methods available at https://www.cbfish.org/Document.mvc/Viewer/P165414; https://www.cbfish.org/Document.mvc/Viewer/P164600. Personal communication with Mari Williams, NOAAF NWFSC/OAI Dec 2019.

Description (what is this a data set of)
contributor,
Citation
methods,
download date
data source(s)
compiler,

Format add info if number
ESU/DPS (constrained field)
Fracwild (decimal number between 0 and 1).
Year. Integer. See details for ESU/DPS comments.
NUMBER_OF_SPAWNERS Total spawners or escapement.

E1 Update VRData with 2020 data

As a member of the Pacific NW Aquatic Monitoring Working Group or contributor to Coordinated Assessments
I want to be able to look up and download the raw data and sources for populations in the 2020 SR
so that I can see where the data came from including the CAX sources, so I can compare the raw counts to my own data on those, and so I can see where there are gaps.

This requires: Get the pentultimate VR 2020 data that went to SPS into the GitHub R package (pkgdown) and fix problems that arose when Mari made all the columns uniform across the ESUs.

• #13
• #14
• #15
  • that way I can test code against old data (backwards compatible) and against new data
• #16
• #17
• #18
• #23
• #26
• S1.8 Push changes to main and delete branches
• S1 new story

S3 Add CITATION file

S1.2 Change 2020 data files to have the updated colnames

• Download SPS
• Read in xls and divide into files
• Write code to read in file, change columns to standard
• write csv to data-raw in branch

Check that metadata includes everything needed to understand

S1.1 Decide on colnames for data files

• Write wiki pages
• Contact Mari and Katie feedback
• Incorporate feedback into wiki

S1 Fork so Chris, Mari and Katie can review

Last step

S1.5 Make sure VRData recognizes new column names

• Add compiler

S3 In Rd files, don't list citation multiple times

Group populations together that share the same citation and source.