uupharmacometrics/xpose

Graphical diagnostics for pharmacometric models

xpose's Issues

Change detection of eta type variables

In read_nm_tables.R function index_table the variable detection for type eta is
stringr::str_detect(.$col, 'ETA\d+|ET\d+') ~ 'eta',

Consider changing to
stringr::str_detect(.$col, '^ETA|^ET') ~ 'eta'

to allow detection of ETA parameters by name e.g.
ETA_CL, ETA_V etc.

prm_distrib Fails when Parameter Has Parentheses

One of my parameters is named "ETA(CL)", and this appears to cause the following error:

prm_distrib(xpdb=xpdb)
Error in ETA(CL) : could not find function "ETA"

The traceback() suggests it's an issue with attempting to map a column name to a function for summarization:

> traceback()
18: .f(.x[[i]], ...)
17: map(.x[sel], .f, ...)
16: map_if(ind_list, is_helper, eval_tidy)
15: vars_select_eval(.vars, quos)
14: tidyselect::vars_select(names(.data), !!!quos(...))
13: select.data.frame(.data, !!!dots)
12: select(.data, !!!dots)
11: select_.data.frame(., .dots = cols)
10: dplyr::select_(., .dots = cols)
9: function_list[[i]](value)
8: freduce(value, `_function_list`)
7: `_fseq`(`_lhs`)
6: eval(quote(`_fseq`(`_lhs`)), env, env)
5: eval(quote(`_fseq`(`_lhs`)), env, env)
4: withVisible(eval(quote(`_fseq`(`_lhs`)), env, env))
3: get_data(xpdb, .problem = .problem) %>% dplyr::select_(.dots = cols) %>% 
       dplyr::select_if(.predicate = function(x) length(unique(x)) == 
           1) %>% colnames()
2: drop_fixed_cols(xpdb, .problem, cols = prm_col, quiet = quiet)
1: prm_distrib(xpdb = xpdb)

In my NONMEM model, I'm using:

$ABBR REPLACE ETA(CL)=ETA(2)

grid.arrange not working with dv_vs_pred plots

grid.arrange() output fails to identify the title and subtitle of the individual plots. For example,
p1<-dv_vs_pred(xpdb_run1)
p2<-dv_vs_ipred(xpdb_run2)
grid.arrange(p1,p2, ncol=2) output plot does not have the title and subtitle for the plots.

Multi-page graphs show all pages by default

I would suggest that perhaps multi-page graphs e.g. ind_plots show all pages by default rather than the last page. Setting page = 1:n is fine up to a point, but you need to go to the Help pages to find out how.

Failure to create run summary using NONMEM 7.4

Hello,

I have run the same model (IV 1-CMT) on NM v 7.3 and 7.4. The run summary can be created with v 7.3 but when using v 7.4 I get the following error:

Failed to create run summary. Evaluation error: 'str_trunc' is not an exported object from 'namespace:stringr'.

Possible compatibility issue with v 7.4?

I'm attaching my output file for your review.
run1.txt

kbVHC diagnostic for RTTE models

If xpose would also cover diagnostics for TTE models, the implementation of the kbVHC, a simulation-free diagnostic for RTTE models, might be of interest. https://link.springer.com/article/10.1208/s12248-017-0162-9

get_prm fails to read .ext file when BLOCK matrices are used in OMEGA

For Moxonidine example where there is a BLOCK(2) on CL and V the .ext file has the full OMEGA matrix, but apart from the CL, V block there are no other named OMEGAs in the control stream.
get_prm fails due to the number of named OMEGA items not being matched by the number of columns of OMEGA reported in the .ext file. Could we apply heuristics e.g. if OMEGA(x,y) estimate = 0 then ignore this column in get_prm?

run1.txt

Feature Request: Simplify Recoding Parameters

Within NONMEM, parameters must be numbered, but it would help if we could recode parameters within xpose either by merging with a list of mapping data.frames or by running recode on the data.

As an example, SEX = 1 isn't very informative while SEX = "Female" or SEX = "Male" is (https://uupharmacometrics.github.io/xpose/articles/customize_plots.html#modify-aesthetics).

A simple implementation may make a method for the dplyr::recode() function that would apply to xpdb objects that would also take a column name.

Missing output tables (gradients) not handled cleanly

If *.grd tables are missing they are not read in. However, the prm_vs_iteration and grd_vs_iteration functions do not check to see whether this data has been loaded and fail without an appropriate message if it is missing.

summary(xpdb) doesn't report fully if file is control stream?

When using xpose_data with a NONMEM control stream, xpose_data doesn't seem to pick up and read any associated NONMEM output (.lst) file. So summary(xpdb) provides only minimal information. Is this expected behaviour? xpose_data / read_nm_model identifies the correct table files and .ext etc. but doesn't appear to parse the output file for use in summary(...).

When calling xpose_data using runno or providing an .lst (or equivalent) NONMEM output for the file argument the complete information is parsed from the output.

It might be good for xpose_data to search within the directory of file for output files with the same filename as the control stream if the ext = .mod.

`dplyr::n()` does not work when editing xpdb with dplyr verbs

When editing an xpdb, dplyr::n() fails to return the proper number of rows of the actual dataset.

xpdb_ex_pk %>%
mutate(N = n())

As temporary workaround one could use something like:

xpdb_ex_pk %>%
mutate(N = length(ID))

Any suggestions on how to implement dplyr::n() in xpose are welcome

Column names for NM files

I noticed that the NM files (.ext, .phi) have column names that correspond to the format you would get when reading a file with read.table (instead of readr). So for an .ext file the header in the xpdb is
SIGMA.1.1. OMEGA.1.1. OMEGA.2.1. OMEGA.2.2. OMEGA.3.1. OMEGA.3.2.
rather than
SIGMA(1,1)` `OMEGA(1,1)` `OMEGA(2,1)` `OMEGA(2,2)` `OMEGA(3,1)` `OMEGA(3,2)
First, I thought this is a bug, but then I saw that an explicit replace is done in the code. So I wanted to ask why this was done and whether we shouldn't move to the second version?
In my opinion, the second version is clearer and more in line with modern R.

Handling plots with multiple pages

Find a nice way to handle multiple pages while keeping a ggplot2 object

depreciate xpose_data?

We currently have xpose_data() for nonmem and xpose_data_nlmixr() for nlmixr. For consistency I am thinking of soft depreciating xpose_data() and rename it xpdb_nm() or xpose_nm() (or other if suggestions) we would then get similar naming for nlmixr e.g. xpdb_nlmixr() or xpose_nlmixr(). Any thoughts @andrewhooker @sebastianueckert, @MikeKSmith , @mattfidler?

Avoid testing tibble output verbatim

xpose seems to be testing tibble output verbatim. The output has changed in the upcoming version 1.4.1. Could you please make sure that you only test output that is fully under your control?

https://github.com/tidyverse/tibble/blob/r-1.4.1/revdep/problems.md#xpose

Feature Request: For DV vs PRED (and IPRED), add `square` option

I prefer to see the x- and y-axes of my DV vs. IPRED and DV vs. PRED figures as square with equal ranges.

Would you consider adding an argument like square=TRUE (default FALSE to align with current practice), and if TRUE, it would add the following (which is the best way I know to get equal ranges and square axes):

  geom_blank(mapping=aes(x=DV, y=IPRED)) +
  coord_equal()

Keywords do not display correctly when multiple graphs are arranged within a single plot

When arranging multiple xpose plots in a single figure, e.g. using gridExtra::grid.arrange, the keywords in the titles subtitles and annotations stop working correctly - e.g. title displays as @y vs. @x | @run instead of the actual values.

Is there a workaround for this?

Issue with reading sdtab of non-standard format

Hi I'm getting the folowwing message when running xpose_data:

Warning messages:
1: Dropped: sdtab001b due to unexpected data format 
2: No table imported

I think the problem is that I use FORMAT=SF10.3 in $TABLE in the NONMEM control stream and that this format is not compatible with xpose.

I tried to debug and found that commenting out some code in the read_args() function fixes the issue:

read_args<-
  function (x, quiet, col_types = readr::cols(.default = "d"), 
            na = "NA", comment = "TABLE", skip = 1, ...) 
  {
    top <- x$top[[1]]
    #  if (is.na(top[3]) || !stringr::str_detect(top[3], "\\d+E[+-]\\d+\\s*")) {
    #    warning(c("Dropped: ", basename(x$file), " due to unexpected data format"), 
    #            call. = FALSE)
    #    return(dplyr::tibble(fun = list(), params = list()))
    #  }
    fun <- dplyr::case_when(stringr::str_detect(top[3], "\\d,\\d+E[+-]\\d+\\s*;") ~ 
                              "csv2", stringr::str_detect(top[3], "\\d.\\d+E[+-]\\d+\\s*,") ~ 
                              "csv", TRUE ~ "table")
    skip_h <- dplyr::if_else(stringr::str_detect(top[1], "TABLE NO\\.\\s+\\d"), 
                             1, 0)
    if (!stringr::str_detect(top[1 + skip_h], "[A-z]{2,}+")) {
      warning(c("Dropped: ", basename(x$file), " due to missing headers."), 
              call. = FALSE)
      return(dplyr::tibble(fun = list(), params = list()))
    }
    col_names <- top[1 + skip_h] %>% stringr::str_trim(side = "both") %>% 
      stringr::str_split(pattern = dplyr::case_when(fun == 
                                                      "csv" ~ ",", fun == "csv2" ~ ";", fun == "table" ~ 
                                                      "\\s+")) %>% purrr::flatten_chr() %>% stringr::str_trim()
    dplyr::tibble(fun = read_funs(fun), params = list(list(file = x$file, 
                                                           skip = skip, comment = comment, na = c(na, col_names), 
                                                           col_names = col_names, col_types = col_types,...)))
  }

But of course this is just a hack. Maybe the develpopers find a more elegant solution to allow different formats of NONMEM tables. Please post me if you need the NONMEM controlstream + datafile to reproduce the error.

Regards, Andreas.

*_distrib plots do not show density correctly

When requesting density (type="hdr") for eta_distrib, res_distrib, prm_distrib, and cov_distrib, density overlay is on the wrong scale (very small).

Question: is this new version of xpose includes survival plots and VPC for survival plot?

Hello ,

I just wanted to know if this version includes the a ggplot-based version of kaplan.plot so we can have pretty graphics for survival? and VPC?

Many thanks
Celine

Cannot Load Files when ID Column is Aliased

In my $INPUT block, I specify that "ID=USUBJIDNUMERIC". With that, the ID in the tables is now titled "USUBJIDNUMERIC" rather than "ID".

I think that the fix would be to detect mapping and allow an argument that makes it to the index_table function (

xpose/R/read_nm_tables.R

Lines 366 to 377 in 942f8b1

	dplyr::mutate(type = dplyr::case_when(
	.$col == 'ID' ~ 'id',
	.$col == 'DV' ~ 'dv',
	.$col == 'TIME' ~ 'idv',
	.$col == 'OCC' ~ 'occ',
	.$col == 'DVID' ~ 'dvid',
	.$col == 'AMT' ~ 'amt',
	.$col == 'MDV' ~ 'mdv',
	.$col == 'EVID' ~ 'evid',
	.$col == 'IPRED' ~ 'ipred',
	.$col == 'PRED' ~ 'pred',
	.$col %in% c('RES', 'WRES', 'CWRES', 'IWRES', 'EWRES', 'NPDE') ~ 'res',

) where the different required column types are mapped. Ideally, this would be auto-detected from the $INPUT block and possible to override by the user.

Generally, this can apply for any column since they all could be mapped to different names.

Use existing tools to change plot layers and themes

The ggedit package from Metrum RG (https://github.com/metrumresearchgroup/ggedit) may be useful to control plot layers and themes once the main aesthetics of the plots have been defined.

order of stratified vpc plots

First of all, thanks for a great package! When generating vpc data based on non-PsN generated simulations the order of elements in column "strat" and the columns containing the stratification variables (both in data frame "vpc_dat" and "aggr_obs") seems to be based on data class character. This although the stratification variable in problem 1 and 2 are data class factor, with appropriate levels and ordering. This can results in unexpected ordering of vpc plots when stratification variable is for example 2, 5 and 10, where 10 then will be ordered before 2 and 5. If my understanding is correct, is it possible to not do this intermediate transformation of the stratification variables during vpc data generation or is there any obvious solution/feature that I have missed?

labels not carried in multipanel plot

The labels are populated in the individual figure, but when the plots are combined together, only the templated text is presented. Example code below:

xpdb <- xpose_data(dir=projpath,runno=runno)
p1 <- dv_vs_pred(xpdb)
p2 <- dv_vs_ipred(xpdb)
p3 <- res_vs_idv(xpdb)
p4 <- res_vs_pred(xpdb)

library(gridExtra)
grid.arrange(p1,p2,p3,p4, nrow=2)

Access by `::` seems to not work the same as direct access

Hi @guiastrennec

Again, thanks for the package. I wanted to point out some strange inconsistency when looking at nlmixr's use of xpose

It is probably reproducable with other examples, but I will use nlmixr

library(nlmixr)
library(xpose.nlmixr)
one.compartment.saem <- function() {
    ini({
        tka <- .5   # Log Ka
        tcl <- -3.2 # Log Cl
        tv <- -1    # Log V
        eta.ka ~ 1
        eta.cl ~ 2
        eta.v ~ 1
        add.err <- 0.1
    })
    model({
        ka <- exp(tka + eta.ka)
        cl <- exp(tcl + eta.cl)
        v <- exp(tv + eta.v)
        d/dt(depot) = -ka * depot
        d/dt(center) = ka * depot - cl / v * center
        cp = center / v
        cp ~ add(add.err)
    })
}
fit <- nlmixr(one.compartment.saem, theo_sd, est="saem")

Then xpose.nlmixr creates the xpose object:

xpdb <- xpose_data_nlmixr(fit)

With :: access I get:

>   xpose::dv_vs_ipred(xpdb)
Using data from $prob no.1
Filtering data by EVID == 0
Error: Columns `problem`, `subp`, `label` must be length 1 or 6, not 4, 4, 4

However, with direct access everything is OK:

>   xpose::dv_vs_ipred(xpdb)

Cannot use grid arrange

Hello,

I am trying to use grid arrange so I can put the basic GOF plots all on the same page in a PDF file. However when I do this the plot title such as DV vs. PRED and number of observations disappear and revert to their references , @y vs. @x @run @ofv. Do you know how I can do this without losing this information.

Thanks

Justus

Stratification on 2 variables within vpc_data

Dear Xpose group

I would like to use vpc_data from the xpose package to be able to stratify on 2 variables.
I have 2 columns in the original file as well as in the sdtab, named CMT (the NONMEM compartment) and GRP (the dose group).

With the following code I want to stratify first on GRP, then on CMT:

vpc_data(xpdb, vpc_type = "continuous",
stratify = ~GRP+CMT,
psn_folder = dir")

I get the following output after running the command:

VPC continuous --------------------

1. Gathering data & settings
   Importing PsN generated data
   |============================| 100% 86 MB
   Setting stratifying variable to GRP, CMT

2. Computing VPC data
   Configuring and initializing...
   Stratifying oberved data...
   Error in check_stratification_columns_available(obs, stratify, "observation") :
   The following specified stratification columns were NOT found in observation data:
   GRP,CMT

Did anybody encounter a similar issue? How can it be resolved?

Thank you in advance!

Best
Huybrecht

sdtab is not found

Hi!
I am pretty new to Xpose. I am trying to run Xpose through Pirana interface. When I try to run a model and execute it through NONMEM, it gives me the following message:
"The table file sdtab002 was not found. Please check your output tables." Here 002 refers to a model file that was used.
It is driving me nuts. In the morning, I seemed to figure out the problem for a while but it was an accident and I don't know how I figured it out in the morning. However, I am back to this same problem.
Even when I try to execute model through PsN, it gives me the following message in the command window:
" 'execute' is not recognized as an internal or external command, operable program or batch file.
The table file sdtab002 was not found. Please check your output tables.
Stopping R execution.Error in print(pl) : object 'pl' not found
Execution halted
C:\Users\Sumet\Desktop\PiranaSum\demo\auto_20181025\auto_20181025>"

PLEASE HELP ME!! As I said I am very new to Pirana, NONMEM, R and XPOSE.

Feature Request: Store the .ctl and .lst files (or proivde as.character to get them back)

When writing a report, I like to put in the model listing. It would be helpful to be able to add that listing with code like:

cat(as.character(xpdb$code))

If that is not feasible (or if it is infeasible to identically regenerate the .mod and/or .lst file from the $code, could the xpdb have a new element that stores the model and list files for later use?

Use geom_point for ind_plots if subject has only one obs

PRED and IPRED in an individual plot do not show for subjects that have only one observation as geom_line doesn't draw anything. It could be good to switch to geom_point for those cases or implement a custom geometry.

Error Plotting Poorly Formatted ETA Name

My eta variables are not correctly detected, so I manually assigned them with:

mydata <-
  xpose_data(runno=1, dir="../NONMEM/PK/") %>%
  set_var_types(eta=c("ETA(FGUT)", "ETA(ALAG)", "ETA(KA)", "ETA(CL)", "ETA(QCENT", "ETA(QCENT_1", "ETA(VCENT", "ETA(VP1)", "ETA(VP2)"))

When I then try to plot with:

eta_distrib(mydata)

I get the following error:

> eta_distrib(mydata)
Error in parse(text = x) : <text>:2:0: unexpected end of input
1: ETA(QCENT
   ^

Omega Labels Mis-Detected?

(Hopefully this is unrelated, but I'm using @MikeKSmith 's #113.)

I think that this is an error with name detection of diagonal $OMEGA values.

I have the following $OMEGA block set in a model:

$OMEGA
 0.04   ; IIV KA
$OMEGA BLOCK(2)
 0.14  ; IIV CL
 -0.2  ; COV-IIV CL-V2
 0.4   ; IIV V2
$OMEGA
 0 FIX ; IIV Q3
 0 FIX ; IIV V3
 0 FIX ; IIV Q4
 0 FIX ; IIV V4

With it, I get the error:

> get_prm(xpdb=xpdb)
Returning parameter estimates from $prob no.1, subprob no.0, method lce
... # Some of the parameters here
# ... with 25 more rows
Warning message:
[$prob no.1, subprob no.0, lce] $OMEGA labels did not match the number of OMEGAs in the `.ext` file. 

As I looked, it appears to be an issue with off-diagonal $OMEGA values here such that they have blank names:

xpose/R/xpdb_access.R

Lines 426 to 433 in 74855b2

	n_omega <- sum(prms$type == 'ome' & prms$diagonal, na.rm = TRUE)
	omega_names <- data$prm_names$omega
	if (n_omega != length(omega_names)) {
	warning('[$prob no.', data$problem, ', subprob no.', data$subprob, ', ', data$method,
	'] $OMEGA labels did not match the number of OMEGAs in the `.ext` file.', call. = FALSE)
	} else {
	prms$label[prms$type == 'ome' & prms$diagonal] <- omega_names
	}

At that point, my omega names are:

[1] "IIV KA" "IIV CL" ""       "IIV V2" "IIV Q3" "IIV V3" "IIV Q4" "IIV V4"

while the number of diagonal omegas is accurately detected as 7.

Calculation of PI in VPC

Hello,
First of all thank you for all your work in making this new xpose package for ggplot enthusiasts !

I have an issue with how percentiles are calculated for VPCs prediction intervals. There seems to be a difference between this new package and PsN + xpose4

In PsN for a i-th percentile of the simulations, first pooling the simulated DVs from all the NONMEM simulation runs is done then the percentile is determined on this data.

Here, it seems that the i-th percentile of the simulation, first this percentile is determined separately on DV from each NONMEM simulation run, and then the median of all those calculated percentiles is calculated. That it as least how I interpret the code coming from the vpc package link starting line 250 :

if(verbose) message("Calculating statistics for simulated data...") tmp1 <- sim %>% dplyr::group_by(strat, sim, bin) aggr_sim <- data.frame(cbind(tmp1 %>% dplyr::summarise(quantile(dv, pi[1])), tmp1 %>% dplyr::summarise(quantile(dv, 0.5 )), tmp1 %>% dplyr::summarise(quantile(dv, pi[2])), tmp1 %>% dplyr::summarise(mean(idv)))) aggr_sim <- aggr_sim[,-grep("(bin.|strat.|sim.)", colnames(aggr_sim))] colnames(aggr_sim)[grep("quantile", colnames(aggr_sim))] <- c("q5", "q50", "q95") colnames(aggr_sim)[length(aggr_sim[1,])] <- "mn_idv" tmp <- aggr_sim %>% dplyr::group_by(strat, bin) vpc_dat <- data.frame(cbind(tmp %>% dplyr::summarise(quantile(q5, ci[1])), tmp %>% dplyr::summarise(quantile(q5, 0.5)), tmp %>% dplyr::summarise(quantile(q5, ci[2])), tmp %>% dplyr::summarise(quantile(q50, ci[1])), tmp %>% dplyr::summarise(quantile(q50, 0.5)), tmp %>% dplyr::summarise(quantile(q50, ci[2])), tmp %>% dplyr::summarise(quantile(q95, ci[1])), tmp %>% dplyr::summarise(quantile(q95, 0.5)), tmp %>% dplyr::summarise(quantile(q95, ci[2])), tmp %>%dplyr::summarise(mean(mn_idv)))) `

I guess that the median of the calculated percentiles should be equivalent to the percentile over all the data when the quantity of data tends to infinity but it is different enough that it shows with the model I am working on currently. Was there any specific reason for this change ? Could one method be especially better than the other ?

Thanks a lot !

Last page of individual plots

When I try to make individual plots, 'nrow' and 'ncol' are changed based on the remains of individuals.
For example, if I make an individual plot for 29 subjects with ncol=3 and nrow=3, last page of individual plot is ncol=2, nrow=1. I just want to make a plot for the last page as same with previous page (ncol=3, nrow=3). Can I control this situation?

I used following script.

xpdb <- xpose_data(runno = '117', ext = '.out', prefix="", gg_theme = theme_bw2())
xpdb %>% ind_plots(ncol=3, nrow=3)

Summary Creation Fail

The majority of my runs create the following error: Failed to create run summary. Evaluation error: invalid format '%02d'; use format %f, %e, %g or %a for numeric objects.

Initially I thought it was because I was specifying the format of my output tables in the NM control stream. I removed these specifications and it seemed to rectify the issue until my next run when the same error occurred without specifying the format of the output table. FYI, I usually use the following format in the NM control stream: FORMAT="t1PE14.7". I can still create basic GoF plots for my model but any information pulled from the run summary is incorrect (i.e. plot titles). Possible bug?

Feature Request: Make `list_vars` invisibly return a list of vars

It would be helpful if there were a simple way to get the mapping of column names to parameter type. An interface that seems to be a good fit would be to return the list of vars invisibly from the list_vars() function. (Or to make it return a new object type with a print method that printed like list_vars() currently does.)

Enhancement: line of identity

The line of identity is currently grey by default, which makes it hard to see against the data, which are also grey by default. Suggest the default be changed to another colour, like red or blue.

Handling multiple $PROB within a single run

How should we handle multiple $PROB problems with xpose? Each could potentially have $TABLE statements producing output which could be handled by xpose.

xpose_data reads ID and OCC as factor

Reading ID and OCC as factor makes plotting somewhat easier (ID and OCC don't have to have type changes for each plotting function), but it makes merging with original data somewhat harder, when the user might not be expecting these variables to be of type factor. We only see type mismatch in left_join with original data.

Quick fix: make it clear in documentation for xpose_data that key columns will be of type factor.
Longer term fix: Maybe refactor so that xpdb data objects have integer type (based on readr parsing of column type) and move factor type change to plotting functions?

"dir" argument ignored in read_nm_model if "file" is specified

When specifying both dir and file in read_nm_model (and in xpose_data), the dir argument seems to be ignored in line 52: model <- readr::read_lines(file).

So when the user specifies:
xpose_data(dir="./inst/extdata/", file="run001.lst")
we get the error:
Error: File run001.lst not found.

Fix issues with new version of purrr

by_row, by_slice and slice_rowsjust got depreciated from purrr. They should be replaced by nest(), and pmap().

ETA distributions in nlmixr

Hi,

I'm having an issue with plotting ETAs generated by nlmixr using eta_distrib:

Error in mutate_impl(.data, dots) : 
  Column `variable` must be length 3 (the group size) or one, not 36

There are 3 ETAs in the model, and input to the function is an Xpose data object created using the xpose_data_nlmixr function in xpose.nlmixr. The whole example can be recreated using the example code at https://github.com/nlmixrdevelopment/nlmixr/blob/master/vignettes/running_nlmixr.Rmd.

I was able to trace the issue back to a call to fetch_data in xplot_distrib, but I got stuck at that point. I'm not enough of a dplyr expert to figure out what's happening: do you have any insights? It should be a really simple thing to do, and as far as I can see the Xpose data object created by xpose_data_nlmixr is identical to an equivalent created by xpose_data... or am I missing something obvious?

Here's the offending Xpose data object:
nlmixrFit.zip

parse_title no longer works with the most recent version of the tidyverse.

Hi @guiastrennec ,

With the most recent tidyverse/ggplot2 (v 3) the parse_title no longer works as expected. Here is the theo example using xpose.nlmixr theophille data.

library(nlmixr)
library(xpose)
library(xpose.nlmixr)

theoSd <- function() {
    ini({
        tka <- .5
        tcl <- -3.2
        tv <- -1
        eta.ka ~ 1
        eta.cl ~ 2
        eta.v ~ 1
        add.err <- 0.1
    })
    model({
        ka <- exp(tka + eta.ka)
        cl <- exp(tcl + eta.cl)
        v <- exp(tv + eta.v)
        linCmt() ~ add(add.err)
    })
}

theoFit <- nlmixr(theoSd, theo_sd, est="saem")

xpdb <- xpose_data_nlmixr(theoFit)
dv_vs_pred(xpdb)

When you run dv_vs_pred you get:

> dv_vs_pred(xpdb)
Using data from $prob no.1
Filtering data by EVID == 0
Warning message:
y, x is not part of the available keywords. Check ?template_titles for a full list. 
> 

I'm not sure which of the tidyverse packages is broken. But you could do the same thing using Base R. However tidyverse fans don't often want to do that, so I will let you decide.

I'm copying @kestrel99 for awareness.

Here is my sessionInfo:

> sessionInfo()
R version 3.5.1 (2018-07-02)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 15063)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252 
[2] LC_CTYPE=English_United States.1252   
[3] LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                          
[5] LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] bindrcpp_0.2.2     xpose.nlmixr_0.1.2 xpose_0.4.3        ggplot2_3.0.0     
[5] nlmixr_0.9.1-3    

loaded via a namespace (and not attached):
 [1] tidyselect_0.2.4          purrr_0.2.5              
 [3] lattice_0.20-35           colorspace_1.3-2         
 [5] RxODE_0.7.2-5             vpc_1.0.2                
 [7] rlang_0.2.1               R.oo_1.22.0              
 [9] pillar_1.2.3              glue_1.2.0               
[11] withr_2.1.2               R.utils_2.6.0            
[13] tweenr_0.1.5              bindr_0.1.1              
[15] plyr_1.8.4                stringr_1.3.1            
[17] munsell_0.5.0             n1qn1_6.0.1-2            
[19] gtable_0.2.0              R.methodsS3_1.7.1        
[21] devtools_1.13.6           mvnfast_0.2.5            
[23] memoise_1.1.0             labeling_0.3             
[25] inline_0.3.15             knitr_1.20               
[27] RcppArmadillo_0.8.600.0.0 rex_1.1.2                
[29] curl_3.2                  parallel_3.5.1           
[31] Rcpp_0.12.17              scales_0.5.0             
[33] jsonlite_1.5              SnakeCharmR_1.0.7        
[35] brew_1.0-6                ggforce_0.1.3            
[37] digest_0.6.15             stringi_1.1.7            
[39] dplyr_0.7.6               lbfgs_1.2.1              
[41] grid_3.5.1                tools_3.5.1              
[43] magrittr_1.5              dparser_0.1.8            
[45] lazyeval_0.2.1            tibble_1.4.2             
[47] PreciseSums_0.3           tidyr_0.8.1              
[49] pkgconfig_2.0.1           MASS_7.3-50              
[51] Matrix_1.2-14             assertthat_0.2.0         
[53] httr_1.3.1                R6_2.2.2                 
[55] units_0.6-0               nlme_3.1-137             
[57] git2r_0.22.1              compiler_3.5.1           
> 

To me the problem is with the following code bit:

keyword <- string %>% 
    stringr::str_extract_all('@[[:alnum:]]+') %>% 
    purrr::flatten_chr() %>% 
    stringr::str_replace(stringr::fixed('@'), '') %>% 
    subset(!. %in% ignore_key)

licensing

Rather than pollute the other PR I think this discussion can be moved here.

The point is we can call the license a LGPL license, however the hard dependencies on ggplot2/ggforce and many of the other packages across the ecosystem are going to end up causing this to reasonably fall under GPL license rather than LGPL.

This is somewhat of a silly discussion in the R ecosystem, as the source code is all available anyway, rather than than problems like deploying as a static binary.

I just wanted to bring up awareness, with regards to future decisions about the ability to include/exclude code and the impact on the project license.

https://en.wikipedia.org/wiki/GNU_General_Public_License#Linking_and_derived_works

https://softwareengineering.stackexchange.com/questions/342156/can-i-use-some-gpl-code-in-lgpl-project

https://opensource.stackexchange.com/questions/1640/if-im-using-a-gpl-3-library-in-my-project-can-i-license-my-project-under-mit-l

http://www.gnu.org/licenses/gpl-faq.html

Overall, the easiest way I've seen GPL licensing described is that if your program cannot function without the GPL'd software, it is pretty clear to be a derivative work, and therefore falls under the license. ggplot2 clearly falls under this purview. That said, realistically, I am unaware of any litigation surrounding R package licensing, so this is likely all a moot point anyway.

Feature Request: Add left_join Method to Augment Data

Similar to #117, it is sometimes helpful to augment the underlying data for plotting purposes.

Could a left_join() method be added that would allow that augmentation such as the following?

left_join.xpose_data <- function(x, y, by=names(y)[1], copy=FALSE, suffix=c(".x", ".y")) {
  ret <- x
  for (data_idx in seq_along(ret$data$data)) {
    if (all(by %in% names(ret$data$data[[data_idx]]))) {
      ret$data$data[[data_idx]] <-
        left_join(x=ret$data$data[[data_idx]], y=y, by=by, copy=copy, suffix=suffix)
    } else {
      warning("Column(s) ", paste(by, collapse=", "), " does not exist in data for problem ", data_idx)
    }
  }
  class(ret) <- class(x) # I think that this is needed because the xpose_data class doesn't have a $<- method
  ret
}

The only limitation I see with this code is that it doesn't assign variable types (and, it's an important limitation).

`dv_preds_vs_idv` with faceting

Hi

I am trying to use the dv_preds_vs_idv with faceting, but I am not getting expected results. Function call without the faceting gives expected results, i.e.,

dv_preds_vs_idv(xpdb, group = "ID") generates a grid of plots (1 row and 3 columns) of DV, IPRED and PRED with independent variable TIME.

With dv_preds_vs_idv(xpdb, group = "ID", facets = c("STUDY")), I expected a 12 plots (I have 4 studies), but I am getting only 4 plots (one for each study) with (I think) DV, IPRED and PRED combined in a single plot for each study. I reported the issue to my colleague Mike Smith also, who suggested to file a issue.

I will be willing to provide code/data privately.

Last, but not the least, many thanks for the package!

Feature Request: Have list_vars return a list of the variables

Having list_vars return a human-usable list of variables is helpful. But in times when I'm reporting on many models simultaneously, it would help to be able to check the var list to see if something is defined. As an example, I'd like to be able to do something like the following in my code:

if (length(list_vars(my_xpdb)$ipred) == 0) {
  my_xpdb <- set_var_types(my_xpdb, ipred="CONC")
}

`filter` function not assigning `xpose_data` class to the return object, issue with recent tidyverse?

I recently updated the tidyverse package and found that filter function seems to return object without xpose_data class assigned.
It can be worked around it by manually adding xpose_data class to the returned object but it would be nice if we don't need to.

Comparing session info though, I'm not sure what is causing this, as dplyr or rlang versions are the same. Unlikely but possibly the recent ggplot2 major update had something to do with it?

Let me know if you need any other information.

a <-  filter(xpdb, ARM==1)

class(a)
#[1] "uneval"

str(a)
# List of 7
# $ code    :Classes ‘nm_model’, ‘tbl_df’, ‘tbl’ and 'data.frame':	746 obs. of  5 variables:
#   ..$ problem   : int [1:746] 0 0 1 1 1 1 1 1 1 1 ...
# ..$ level     : int [1:746] 0 0 1 2 2 3 3 4 4 4 ...
# ..$ subroutine: chr [1:746] "oth" "oth" "pro" "inp" ...
# ..$ code      : chr [1:746] "Fri 08/31/2018 " "03:43 PM" " "ID ARM TIME DV TRT STUD" ...
# ..$ comment   : chr [1:746] "" "" "" "" ...
# ..- attr(*, "file")= chr "run33.lst"
# ..- attr(*, "dir")= chr "run33"
# ..- attr(*, "software")= chr "nonmem"
# $ summary :Classes ‘tbl_df’, ‘tbl’ and 'data.frame':	30 obs. of  5 variables:
# ...

Here is the current session info:

# sessionInfo()
# R version 3.4.3 (2017-11-30)
# Platform: x86_64-w64-mingw32/x64 (64-bit)
# Running under: Windows 7 x64 (build 7601) Service Pack 1
# 
# Matrix products: default
# 
# locale:
# [1] LC_COLLATE=English_United States.1252 
# [2] LC_CTYPE=English_United States.1252   
# [3] LC_MONETARY=English_United States.1252
# [4] LC_NUMERIC=C                          
# [5] LC_TIME=English_United States.1252    
# 
# attached base packages:
# [1] stats     graphics  grDevices utils     datasets  methods  
# [7] base     
# 
# other attached packages:
# [1] bindrcpp_0.2.2  lubridate_1.7.4 forcats_0.3.0   stringr_1.3.1  
# [5] dplyr_0.7.4     purrr_0.2.4     readr_1.1.1     tidyr_0.8.0    
# [9] tibble_1.4.2    tidyverse_1.2.1 xpose_0.4.2     ggplot2_3.0.0  
# [13] usethis_1.3.0   testthat_2.0.0  devtools_1.13.5
# 
# loaded via a namespace (and not attached):
# [1] tidyselect_0.2.4 reshape2_1.4.3   haven_1.1.1      lattice_0.20-35 
# [5] colorspace_1.3-2 yaml_2.1.19      rlang_0.2.0      pillar_1.2.2    
# [9] foreign_0.8-69   glue_1.2.0       withr_2.1.2      tweenr_0.1.5    
# [13] modelr_0.1.2     readxl_1.1.0     bindr_0.1.1      plyr_1.8.4      
# [17] munsell_0.4.3    gtable_0.2.0     cellranger_1.1.0 rvest_0.3.2     
# [21] psych_1.8.4      memoise_1.1.0    knitr_1.20       parallel_3.4.3  
# [25] broom_0.4.4      Rcpp_0.12.16     udunits2_0.13    scales_0.5.0    
# [29] backports_1.1.2  jsonlite_1.5     mnormt_1.5-5     ggforce_0.1.1   
# [33] hms_0.4.2        digest_0.6.15    stringi_1.2.2    grid_3.4.3      
# [37] cli_1.0.0        tools_3.4.3      magrittr_1.5     lazyeval_0.2.1  
# [41] crayon_1.3.4     pkgconfig_2.0.1  MASS_7.3-47      xml2_1.2.0      
# [45] assertthat_0.2.0 httr_1.3.1       rstudioapi_0.7   R6_2.2.2        
# [49] units_0.5-1      nlme_3.1-131     compiler_3.4.3  

Here is the previous session info when the code was working:

# > sessionInfo()
# R version 3.4.3 (2017-11-30)
# Platform: x86_64-w64-mingw32/x64 (64-bit)
# Running under: Windows 7 x64 (build 7601) Service Pack 1
#
# Matrix products: default
#
# locale:
# [1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252   
# [3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C                          
# [5] LC_TIME=English_United States.1252    
#
# attached base packages:
# [1] stats     graphics  grDevices utils     datasets  methods   base     
# other attached packages:
# [1] bindrcpp_0.2.2  xpose_0.4.2     lubridate_1.7.4 forcats_0.3.0   stringr_1.3.1  
# [6] dplyr_0.7.4     purrr_0.2.4     readr_1.1.1     tidyr_0.8.0     tibble_1.4.2   
# [11] ggplot2_2.2.1   tidyverse_1.2.1 usethis_1.3.0   testthat_2.0.0  devtools_1.13.5
#
# loaded via a namespace (and not attached):
# [1] tidyselect_0.2.4 reshape2_1.4.3   haven_1.1.1      lattice_0.20-35 
# [5] colorspace_1.3-2 yaml_2.1.19      rlang_0.2.0      pillar_1.2.2    
# [9] foreign_0.8-69   glue_1.2.0       withr_2.1.2      tweenr_0.1.5    
# [13] modelr_0.1.2     readxl_1.1.0     bindr_0.1.1      plyr_1.8.4      
# [17] munsell_0.4.3    gtable_0.2.0     cellranger_1.1.0 rvest_0.3.2     
# [21] psych_1.8.4      memoise_1.1.0    labeling_0.3     knitr_1.20      
# [25] parallel_3.4.3   broom_0.4.4      Rcpp_0.12.16     udunits2_0.13   
# [29] scales_0.5.0     backports_1.1.2  jsonlite_1.5     mnormt_1.5-5    
# [33] ggforce_0.1.1    hms_0.4.2        digest_0.6.15    stringi_1.2.2   
# [37] grid_3.4.3       cli_1.0.0        tools_3.4.3      magrittr_1.5    
# [41] lazyeval_0.2.1   crayon_1.3.4     pkgconfig_2.0.1  MASS_7.3-47     
# [45] xml2_1.2.0       assertthat_0.2.0 httr_1.3.1       rstudioapi_0.7  
# [49] R6_2.2.2         units_0.5-1      nlme_3.1-131     compiler_3.4.3  

Warning messages despite quite=T

Hi Ben,

When loading output from a model that does not have tables, I get a warning even though I am using quite=T. Is this intended behavior?

Seb

vpc no longer required? vpc as a s3 method?

Hi,

I was trying to see where the vpc package is used? Is it no longer used? If so, it can be dropped as a dependency.

Also, I was wondering if you wouldn't mind making the vpc a s3-compatible object. This way in nlmixr I can import your excellent package and then do a vpc simulation on a simple fit object without having to worry about the vpc conflict in etiher xpose or vpc packages