fitGLS_partition with ncores>1

[arives]

fitGLS_partition with ncores>1

[morrowcj]

@arives, I know it has been a while since you opened this issue, but could you elaborate a bit? I get the same results for the following two sets of code (taken from the examples given in fitGLS_partition:

(outA = fitGLS_partition(CLS_coef ~ 1, partmat = pm, data = df, 
                                       nugget = 0, ncores = 1))

and

(outB = fitGLS_partition(CLS_coef ~ 1, partmat = pm, data = df, 
                                       nugget = 0, ncores = 2))

the results of outA$overall (and outB$overall) is

$coefficients
[1] 0.0001780692

$rcoefficients
         [,1]
[1,] 0.999392

$rSSR
[1] NaN

$rSSE
[1] 0.2702243

$Fstat
[1] NaN

$dfs
df1 df2 
  0 666 

$partdims
   npart partsize 
       3      666 

$t.test
              Est         SE     t.stat    pval.t
[1,] 0.0001780692 0.00228522 0.07792214 0.9378978

$covar_coef
            [,1]
[1,] 5.28559e-06

The F-test statistics (and chisqr) aren't computed for the intercept-only model, but this is expected (since the specified model is equivalent to the null model).

There is an annoying warning (unless do.chisqr.tests=FALSE) and some slots are filled with NaN - perhaps that should change. But I haven't been able to reproduce an outright error or failure.

[arives]

Clay,

Sorry for being too brief before. Here is example code and output from macOS. Various changes don't help (e.g., nugget = 0, do.t.test = FALSE). I also did a FORCE install from github to make sure I'm working with the newest version.

It would be really strange if this were an issue only with MacOS, although I have a hard time imagining that this would be the case.

Cheers, Tony

fitGLS_partition(formula = CLS_coef ~ 1, partmat = pm,

•                        data = df, do.t.test = TRUE, ncores = 1)
  

  |==========================================================| 100%

Coefficients:
(Intercept)
0.0001795086

Cross coefficients:
(Intercept)
(Intercept) 0.8083451

Cross-partition statistics:
rSSR rSSE Fstat
NaN 0.08683609 NaN
Warning message:
In fitGLS_partition(formula = CLS_coef ~ 1, partmat = pm, data = df, :
chisqr test not valid when formula == formula0

fitGLS_partition(formula = CLS_coef ~ 1, partmat = pm,

•                        data = df, do.t.test = TRUE, ncores = 3)
  

Error in array(NA, dim = c(npairs, p, p), dimnames = list(NULL, names(object[[1]]$partGLS$coefficients), :
length of 'dimnames' [2] not equal to array extent

[kelewinska]

As far as i know Mac's have different kernel, so some functions could run differently, or not run at all. (?)

[morrowcj]

I was actually having this same issue on Ruusa's computer, so I had her reinstall all the necessary packages and then reinstall remotePARTS. It worked for her but we had to manually install many of the dependencies because R and devtools were having a hard time installing them automatically. I think the main ones we reinstalled were "cli", "Rcpp", and "devtools" plus all of their dependencies (again, sometimes manually when they wouldn't auto update). The R package manager has a lot of problems with file system permissions in my experience so updating packages is a huge pain in the rear. There are often warnings about sub-packages or dependencies not installing properly. I don't know if that's really an acceptable solution or not.

…

________________________________ From: Katarzyna Ewa Lewinska ***@***.***> Sent: Wednesday, November 23, 2022 11:50:16 AM To: morrowcj/remotePARTS ***@***.***> Cc: Clay Morrow ***@***.***>; Assign ***@***.***> Subject: Re: [morrowcj/remotePARTS] fitGLS_partition with ncores>1 (Issue #3) As far as i know Mac's have different kernel, so some functions could run differently, or not run at all. (?) — Reply to this email directly, view it on GitHub<#3 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AGCCE45ROHSXZJFQ7SPIGRLWJZKNRANCNFSM576QQHHQ>. You are receiving this because you were assigned.Message ID: ***@***.***>

[kelewinska]

to ensure stability you could prepare a docker image with a desired environment.

[arives]

Clay,

I just finished reinstalling R and all packages. Still no joy.

Was Ruusa getting this same error message? It seems a strange error if caused by packages not talking nicely to each other.

I'll ask Jamie to try on his Mac.

Tony

[morrowcj]

Thanks for this, guys. Based on Jamie's response (and the fact that the error occurred with Ruusa's machine, I'll keep looking for a solution.

@Erfea, I'd be interested in learning how to prepare a docker image. It was my impression that docker was a linux-only tool. Can Mac and Windows use it as well, or does it require a virtual machine?

[morrowcj]

@arives, what is the output for this code on your machine?

packageVersion("remotePARTS")

[morrowcj]

I've found the problem and I know how to fix it - just going to take some time to work out and code up.

[arives]

Clay, Great. But sorry it’s a hassle.

[arives]

packageVersion("remotePARTS") [1] ‘1.0.0’ I’ve been forcing the install, so I hope this is the most recent.

[kelewinska]

Happy to hear you have found the core of the issue Clay.

regarding docker, it can be installed on all systems, thus provide transferability across platforms. A remotePARTS-specific docker container is on my ToDo list, but it will happen not before spring 2023. Happy to share it once the image will be defined and tested.
Cheers, Kasia