The gfsynopsis from pbs-assess

Make a call on whether to use sdmTMB or INLA for the survey maps

Both are now implemented. It's currently set up to use sdmTMB.

An advantage for INLA is that it's already written up. A disadvantage is that it's a bit of a different paradigm than the current cpue Tweedie model and the TMB model is a spatiotemporal one that incorporates more data for the fixed effects and spatial random effects.

If going with the spatiotemporal one then the writeup will have to be different.

The end result seems to look very similar in most cases.

An advantage to the TMB one is consistency with what we are likely to do with the range shift project model.

Pull in the commercial maturity frequencies by month

The survey ones aren't that informative since they only happen in the summer.

Check why length at maturity and maturity frequency plots are now blank

I must be filtering them out accidentally now.

Add management section for each species

TAC from Greg or from Gf_management?

see the spreadsheet that Greg sent
try Gf_management first

Management system (e.g. TAC, input controlled fishery, daily bag limit)
SARA/COSEWIC

Search for missing elasmobranch commercial data

Reduce the number of maturity categories

Here's an example for yelloweye now that I've included the commercial samples along with the survey samples:

It now becomes obvious that the size of the circles are mostly showing when the samples are being taken rather than the proportions for each month. Perhaps they should be scaled so they are proportions per month rather than proportions across the entire data set?

Also, we talked about this before and I think we should condense the maturity categories. I think Malcolm had some ideas. @elisekeppel , did you ever talk with Malcolm about this or should I get in touch with him?

Set CSAS review date

Redface check with other GF assessment biologists

Hold steering committee meeting with other necessary parties (Rowan, Malcolm, Chris, Dana, Rob, Greg, Lynne, Norm, Maria)

Substitute data for modelled IPHC map

To convert longitude and latitude to UTMs

gfplot:::ll2utm()

Make the data match this format. The important columns are X, Y, combined, survey. You can ignore the rest. This will be inserted as iphc_fits$pred_dat. The combined is the estimate.

I can help actually integrate it.

         X        Y akima_depth depth_scaled depth_scaled2     combined pos bin    survey
1 249.8263 6027.495       380.4    1.9480471     3.7948874 1.010944e-04  NA  NA IPHC FISS
2 231.5991 6028.491       460.9    2.2949877     5.2669684 6.134827e-05  NA  NA IPHC FISS
3 227.8670 6009.485        49.4   -1.7413082     3.0321544 5.964096e+02  NA  NA IPHC FISS
4 191.8953 6030.973       246.9    1.1668254     1.3614815 3.343166e-01  NA  NA IPHC FISS
5 210.3277 6029.695       221.3    0.9689821     0.9389263 6.071074e+01  NA  NA IPHC FISS
6 209.3769 6011.210       385.9    1.9739919     3.8966440 1.721172e-03  NA  NA IPHC FISS

Add further survey details

Add equations for relative biomass index trends from surveys
Add in survey intention and stratification/design
Add in list of other current/on-going surveys (more than twice in past 10 years)

Maria?

Make seminar series presentation

Can present as lunch and learn talk or seminar series

Figure out saving .png files from R that match pdflatex copy mode specifications

Otherwise it's very slow to compile the png version document.

Relevant links:
https://tex.stackexchange.com/questions/327011/fast-png-embedding-using-xetex
https://tex.stackexchange.com/questions/178572/caching-includegraphics-image-files-for-speed/178611#178611

My current solution is:

library(parallel)
library(foreach)
cores <- parallel::detectCores()[1L]
cl <- parallel::makeCluster(cores)
doParallel::registerDoParallel(cl)

setwd("report/figure-pages")
fi <- list.files(".", "*.png")
out <- foreach::foreach(i = fi) %dopar% {
  system(paste0("optipng -strip all ", i))
}

but this takes time as well and although it makes things faster it still doesn't access 'copy mode' in pdflatex.

Check CSAS library for all B species

Most won't have anything but we should check.

Need to be added to report/report-rmd/bib/spp-refs.bib and report/report-rmd/spp-refs.csv.

get_data(path="data-cache2/") is extracting for all species

get_data() has default type="A" but it seems to be extracting the type B species also.

I think it's something to do with get_spp_names(). Not a big problem, just worth noting that the default seems to get ignored.

I'm setting path="data-cache2/" because that's what appears to be in the make.R file that looks to be used.

Partly documenting this here so I can add it to a readme somewhere for future users (i.e. steps to produce the synopsis report), which I couldn't see anywhere obvious (sorry if I missed it). Also, my laptop is going to restart soon and so I want to document my commands.

Adapt the cpue write up from Pacific Cod for the synopsis report

May want to demonstrate the various components for an example species.

May also want to show the residuals for a selection of species.

Also include that figure that demonstrates the shape of the Tweedie.

Switch out HBLL INS S for sablefish survey for sablefish?

Insert the stratified random trap-based survey that has been going on since 2003?

Add to caveats section

A running list of things to add:

these aren't necessarily the best representations of catch and they won't necessarily match reconstructed timeseries in stock assessments
longline discard weights are not included (and make sure these are all removed)
Some recent species-specific declines in commercial catch, in particular Bocaccio and Yelloweye, are due to implementation of management measures. Might be worth noting as a caveat in the description of these plots.

Bring over previous SQL query tables (in .Rnw) to .Rmd

Probably not worth fighting with the tables and instead doing this in bullets. Probably need to flesh out the text a little bit too.

Create french version of main pages

Check if dogfish longline surveys are in biological sample data already

Check whether we are already grabbing the dogfish longline samples and, if not, can we?

the biological samples were ‘opportunistic’ for research and not for assessment... uncertain if they can be tied to a Fishing Event still.

Add an alphabetical index

Will require some trickery with Markdown section references in R. Too hard to find species of interest when listed by species code.

Make sure all H&L and trap(?) has zero discard kg in catches.R

report year when trawl discards become reliable (1996)

Set two species codes back again when figure out why it breaks

california grenadier is 257
deacon rockfish is 429

Have set these both to NA else I get the error

x Building figure pages for california grenadier
Fitting model for the survey SYN QCS
Interpolating depth to fill in missing data if needed...
Preloading interpolated depth for prediction grid...
Error: $ operator is invalid for atomic vectors

which we thought would be fixed by Sean's 998431.

Am pushing my commits now - have it working okay.

Update the function web diagram with the current package

Notes for writing up IPHC analyses

Just came across this in my notes, probably worth referencing:

1.1.11 OTHER DFO ASSESSMENTS THAT USED IPHC SURVEY DATA

Spiny Dogfish, 2011. Doesn’t look like they looked at effective skates, and somehow joined up
the years prior to 2003 to those after, without any discussion about it.

Quillback Rockfish, 2012. Used survey, with Marie-Etienne’s method. Not sure yet how they went
back before 2003.

Ask Lynne which others may have used it.

Fit HBLL spatial models to N and S separately

Currently combined.

@elisekeppel could you separate grid into N and S?

Best way to install gfsynopsis (etc.) into library?

load_all() works fine, but I get is really for development.

I did install() but it then tried to re-install things like dplyr (that I just updated last night), couldn't do them because I had dplyr loaded in another R session, and then failed. Then dplyr wouldn't work at all so I had to re-install it manually.

I'm wondering about

install(dependencies = FALSE, upgrade_dependencies = FALSE)

but for now am just using load_all().
Thanks!

Add survey location maps

Check capitalization in bibliography

.csl files treat capitalization a bit differently than bibtex. There are many more words that need to be protected now.

Integrate Andy's IPHC standardization going back further

Could wait until next year if needed.

Consider H&L CPUE trends (future plan likely)

Maybe in future?
Better for some spp. than trawl.

Add section on Merged_Catch methods

Add section on future plans

ie. Andy's IPHC historic time series
Mention revision schedule here

Would like to build gfsynopsis in next couple of days.

I think you said you were working some stuff out, and so not to try yet. And that you would give me the model outputs so I don't have to run them, yes?

On #31 it look like I had it building on Aug 13th, apart from a few things that will hopefully get solved when starting over.

I need to put some functions into gfplot, but it will be good to use gfsynopsis at the same time to check everything.

Consider splitting yelloweye and lingcod into inside/outside stocks

Perhaps quillback too, but in same assessment.

Probably easiest to modify data after import to split into two 'species' and then create pages as per usual.

Implement suggested plot tweaks from working group meeting

Send draft synopsis report to CSAS office

For review and determination of captioning/translation.

Send to CSAP to send to CSAS.

Add links to res docs / SARS?

SP:

Is it possible to include hyperlinks to paper in federal science library? I compiled a list that may be of use.

see the spreadsheet that Greg sent for a good starting point.

Finish survey spatial modelling methods

Check ordering of all last names from waves database

Get pacharv3 in gffos table?

Change to call sql queries based on species code, not name

Add section in methods on age/length freq. weighting

refer elsewhere... actually probably mention that we aren't doing it here but may incorporated in the future... and that the code is already written and incorporated

Add timeline for revisiting and updates.

Look over report production suggested every three years, with minor fixes in between.

Add in elasmobranch sample data

Get data into GFBio

Look into missing sablefish catch pre 1980

BC:

... the commercial catch plots which suggest there was no trap or hook and line catch pre-1979; there should be a small amount of domestic hook and line and trap catch back to the mid 60s and early 70s, respectively. There was also substantial foreign hook and line catch 1965-1980 (5-6000 tons).

Fix in merged catch table?

Note lack of foreign in resdoc

Figure out what’s causing the hot pixels in some maps

I’ll bet it’s a bad interpolation from the bathymetry layer onto the survey grid. Could try a different algorithm or even the new high resolution version put out by DFO recently. Needs to match the survey polygon. Also try in log vs not log space?

Finish CPUE standardization methods

include quantile binning strategy
Add example coefficient plot and example jackknife plot
add the simulation testing example that also illustrates coverage

Rewrite spatial methods to use TMB model

Current text explains INLA with delta model. Otherwise nearly identical.

Make sure to explain some of the extra filtering for plotting reasons. And how the depth covariates were centred and scaled.

Can't build document

May as well use this to keep track of stumbling blocks.

Running the

system.time({
  for (i in seq_along(spp$species_common_name)) {
  fig_check <- paste0(file.path("report", "figure-pages"), "/",
    gfsynopsis:::clean_name(spp$species_common_name[i]))
  fig_check1 <- paste0(fig_check, "-1.png")
  fig_check2 <- paste0(fig_check, "-2.png")
  .....

loop of make.R and get the following:

Building figure pages for north pacific spiny dogfish 
Determining qualified fleet for area 3[CD]+|5[ABCDE]+.
Fitting standardization model for area 3[CD]+|5[ABCDE]+.
Fitting CPUE model ...
Getting sdreport ...
Error in gzfile(file, mode) : cannot open the connection
In addition: Warning messages:
1: No data available. 
2: No data available. 
3: In if (!is.na(sc[[1]])) sc <- sc %>% filter(year >= 2003) :
  the condition has length > 1 and only the first element will be used
4: In if (!is.na(sb)) { :
  the condition has length > 1 and only the first element will be used
5: In gzfile(file, mode) :
  cannot open compressed file 'report/cpue-cache/north-pacific-spiny-dogfish-3[CD]+|5[ABCDE]+-model.rds', probable reason 'Invalid argument'
Timing stopped at: 305.7 9.53 316.1
>

My report\cpue-cache\ folder is there but is empty (related to the fifth warning).

pbs-assess / gfsynopsis Goto Github PK

gfsynopsis's People

Contributors

Stargazers

Watchers

Forkers

gfsynopsis's Issues

Recommend Projects

Recommend Topics

Recommend Org