summBg() 'when' argument bugs (with/without 'show.rates' = TRUE)

[sashahafner]

summBg() 'when' argument bugs (with/without 'show.rates' = TRUE)

[sashahafner]

Message from Alex:

When setting these parameters : show.obs = TRUE, show.rates = TRUE, some value in the dataframe are different than when I set show.obs = TRUE, when = "meas", show.rates = FALSE.

In the first case, values from columns cvCH4, cvCH4.tot, cvCH4.inoc, se.inoc are all bigger than the one output from the second settings. I think that in the first case, CH4 volumes are not normalised by VS mass, and in the second they are.

Is it something wanted? Anyway, I find it very confusing…

I could reproduce this problem:

library(biogas)

data(vol2)

data(comp2)

data(setup2)

cbg <- cumBg(vol2, comp = comp2, temp = 20, pres = 1,

         id.name = "bottle", time.name = "days",

         dat.name = "meas.vol", comp.name = "CH4.conc",

         extrap = TRUE)

s1 <- summBg(vol = cbg, setup = setup2, id.name = "bottle",

         time.name = "days", descrip.name = "description",

         inoc.name = "Inoculum", inoc.m.name = "inoc.mass",

         norm.name = "sub.vs", show.obs = TRUE, show.rates = TRUE)

s2 <- summBg(vol = cbg, setup = setup2, id.name = "bottle",

         time.name = "days", descrip.name = "description",

         inoc.name = "Inoculum", inoc.m.name = "inoc.mass",

         norm.name = "sub.vs", show.obs = TRUE, when = "meas", show.rates = FALSE)

head(s1$cvCH4 / s2$cvCH4)

head(s1$cvCH4.tot / s2$cvCH4.tot)

head(s1$cvCH4.inoc / s2$cvCH4.inoc)

head(s1$se.inoc / s2$se.inoc)

[jonasoh]

I have also noticed this behavior (i.e., not getting normalized values returned). As I understand it, it is caused by this block, which is executed if both show.rates and pdwhen are TRUE (and if length(when) == 1, but this is already accounted for by pdwhen IIUC):

biogas/R/summBg.R

Lines 466 to 470 in 0d31b2b

	# Return observations here (early to avoid problem in next 2 blocks--see error messages)
	if(show.rates) {
	summ1 <- summ1[order(summ1[, id.name], summ1[, time.name]), ]
	return(summ1)
	}

I do not get any error messages when commenting out this if-block, but I also do not get the expected data (show.obs is ignored), and am struggling a bit with understanding the reason for this logic.

If nothing else, as a quick "fix", maybe it would make sense to add a warning that normalization is not performed if norm.name is not NULL when returning at this stage?

[sashahafner]

Thanks for the info and suggestion. I'm just getting to this. The summBg() function has gotten quite complicated. I think it suffers from too many little additions over a long period.

Here are some example, starting out with the original problems identified by Alex at the top. I"m using version 1.27 of the package.

library(biogas)
head(feedSetup)

cpw <- calcBgVol(feedVol, comp = 1, temp = 0, pres = 1, data.struct = "wide",
                       time.name = "time.d", vol.name = "1", dry = TRUE, interval = FALSE)

# Example 1: leave out when argument, use show.obs and show.rates
s1 <- summBg(cpw, feedSetup, id.name = 'id', time.name = 'time.d', descrip.name = 'descrip',
               inoc.name = 'BK', inoc.m.name = 'm.inoc', norm.name = 'm.sub.vs', show.obs = TRUE, show.rates = TRUE)
head(s1)
# Problems: 1. Warning given for 'when' argument even though it is not actually provided in call
#           2. Results are not normalized, inoc contribution not subtracted

# Example 2: same, but include when
s2 <- summBg(cpw, feedSetup, id.name = 'id', time.name = 'time.d', descrip.name = 'descrip',
               inoc.name = 'BK', inoc.m.name = 'm.inoc', norm.name = 'm.sub.vs', when = 'meas', show.obs = TRUE, show.rates = TRUE)
head(s2)
all.equal(s1, s2)
# Problem: just 2 from above

# Example 3: now show.obs = FALSE
s3 <- summBg(cpw, feedSetup, id.name = 'id', time.name = 'time.d', descrip.name = 'descrip',
               inoc.name = 'BK', inoc.m.name = 'm.inoc', norm.name = 'm.sub.vs', when = 'meas', show.rates = TRUE)
head(s3)
all.equal(s1, s3)
# Problem: 2. 2 from above
#          3. Missing message about no calculation of means because show.rates = TRUE

# Example 4: try to get means, drop show.rates
s4 <- summBg(cpw, feedSetup, id.name = 'id', time.name = 'time.d', descrip.name = 'descrip',
               inoc.name = 'BK', inoc.m.name = 'm.inoc', norm.name = 'm.sub.vs', when = 'meas')
head(s4)
# No problems

[sashahafner]

So, looking at these results . . .
This original problem still exists:

1. Gives warning on ignoring 'when' for s1 (even though no when is given.
   That can be fixed.

This lack of normalization is still a problem:

2. s1 seems to normalize CH4 volume but s2 does not.

and

I have also noticed this behavior (i.e., not getting normalized values returned).

But neither s1 or s2 are normalized. I like your suggestion @jonasoh to address this issue:

If nothing else, as a quick "fix", maybe it would make sense to add a warning that normalization is not performed if norm.name is not NULL when returning at this stage?

Could this be a permanent fix (more than a quick fix)? When show.rates is used, does the user need normalized CH4 yields?

[sashahafner]

I tried to sort out the messages in the latest commit 20b2488. You can get it with:

devtools::install_github('sashahafner/biogas', ref = 'dev')

See below for current behavior.

> cpw <- calcBgVol(feedVol, comp = 1, temp = 0, pres = 1, data.struct = "wide",
+                        time.name = "time.d", vol.name = "1", dry = TRUE, inter
val = FALSE)
Only a single value was provided for biogas composition (1), so applying it to a
ll observations.
Using a standard pressure of 1 atm and standard temperature of 0 C for standardi
zing volume.
> # Example 1: leave out when argument, use show.obs and show.rates
> s1 <- summBg(cpw, feedSetup, id.name = 'id', time.name = 'time.d', descrip.nam
e = 'descrip',
+                inoc.name = 'BK', inoc.m.name = 'm.inoc', norm.name = 'm.sub.vs
', show.obs = TRUE, show.rates = TRUE)
Response variable (volume) is cpw$cvCH4.
Inoculum contribution subtracted based on setup$m.inoc.
Returning output with calculated relative production rates.
Warning message:
In summBg(cpw, feedSetup, id.name = "id", time.name = "time.d",  :
  Volume values were *not* normalized!
  To get normalized values, run with show.rates = FALSE (default).
> # Example 2: same, but include when
> s2 <- summBg(cpw, feedSetup, id.name = 'id', time.name = 'time.d', descrip.nam
e = 'descrip',
+                inoc.name = 'BK', inoc.m.name = 'm.inoc', norm.name = 'm.sub.vs
', when = 'meas', show.obs = TRUE, show.rates = TRUE)
Response variable (volume) is cpw$cvCH4.
Inoculum contribution subtracted based on setup$m.inoc.
Returning output with calculated relative production rates.
Warning messages:
1: In summBg(cpw, feedSetup, id.name = "id", time.name = "time.d",  :
  You set "show.rates = TRUE", so "when" argument will be ignored.
2: In summBg(cpw, feedSetup, id.name = "id", time.name = "time.d",  :
  Volume values were *not* normalized!
  To get normalized values, run with show.rates = FALSE (default).
> # Example 3: now show.obs = FALSE
> s3 <- summBg(cpw, feedSetup, id.name = 'id', time.name = 'time.d', descrip.nam
e = 'descrip',
+                inoc.name = 'BK', inoc.m.name = 'm.inoc', norm.name = 'm.sub.vs
', when = 'meas', show.rates = TRUE)
Response variable (volume) is cpw$cvCH4.
Inoculum contribution subtracted based on setup$m.inoc.
Returning output with calculated relative production rates.
Warning messages:
1: In summBg(cpw, feedSetup, id.name = "id", time.name = "time.d",  :
  You set "show.rates = TRUE", so "when" argument will be ignored.
2: In summBg(cpw, feedSetup, id.name = "id", time.name = "time.d",  :
  Volume values were *not* normalized!
  To get normalized values, run with show.rates = FALSE (default).
3: In summBg(cpw, feedSetup, id.name = "id", time.name = "time.d",  :
  Means are not calculated!
> # Example 4: try to get means, drop show.rates
> s4 <- summBg(cpw, feedSetup, id.name = 'id', time.name = 'time.d', descrip.nam
e = 'descrip',
+                inoc.name = 'BK', inoc.m.name = 'm.inoc', norm.name = 'm.sub.vs
', when = 'meas')
Response variable (volume) is cpw$cvCH4.
Inoculum contribution subtracted based on setup$m.inoc.
Response normalized by setup$m.sub.vs.
>

[sashahafner]

What do you think @jonasoh ?

[jonasoh]

This looks much clearer to me now, good job. If one needs both rates and normalized values, it is also now clear how to get those.

[sashahafner]

Great! Closing.