I can't reproduce your issue posted on SO.

[pbreheny]

I can't reproduce your issue posted on SO.

[YaohuiZeng]

Hi Florian,

Thank you very much for taking a look. I just committed the changes. I suspect the error is due to the behavior of "XPtr", since there are some key words "PreserveStorage", "standard_delete_finalizer", etc.

__Z11get_elem_bmIdEdN4Rcpp4XPtrI9BigMatrixNS0_15PreserveStorageEXadL_ZNS0_25standard_delete_finalizerIS2_EEvPT_EEEEddii

[privefl]

You don't have any R/RcppExports.R and src/RcppExports.cpp?

It seems you're making your package "the old way". How do you compile and install your package?

[YaohuiZeng]

No, I don't. I used to have RcppExports.cpp, but later combined them into .cpp files. RcppExports.R file just include wrapper R functions for .Call(). The previous version (before adding templates) works well. So I am not sure whether having those RcppExports files make a difference.

I compile and install the package using R CMD commands.

[privefl]

I've tried to pass the accessor with the pointer as argument as it is done there.
You can compare the codes there.
But it doesn't seem to change a thing.

One thing I don't understand is why your utilities.cpp has

// This file was generated by Rcpp::compileAttributes
// Generator token: 10BE3573-1514-4C36-9D1C-5A225CD40393

while it seems to be modified by hand.

Moreover, I'm currently on Windows, which doesn't help with the debugging.

[YaohuiZeng]

Thanks for giving a try! I just did exactly the same thing, and failed too.

I think the return type of functions like cd_gaussian_fit_cpp() should be Rcpp::List, not SEXP. So I changed that, but didn't help.

I also followed the link you pointed out but NOT passing matrix accessor as an argument, instead calling function like BigColSums(xpMat) and it WORKED! (See code below, that code works!!!!!!!)

So really confused about where the bug is!!!!!!!!!

// Logic for BigColSums.
template <typename T>
vector<int> BigColSums2(XPtr<BigMatrix> pMat) {

  // Create the vector we'll store the column sums in.
  MatrixAccessor<T> mat(*pMat);
  vector<int> colSums(pMat->ncol());
  for (size_t i=0; i < pMat->ncol(); i++) {
    for (size_t j=0; j < pMat->ncol(); j++) {
      colSums[i] += mat[i][j];
    }
  }
  return colSums;
}

// Dispatch function for BigColSums
//
// [[Rcpp::export]]
NumericVector BigColSums(SEXP pBigMat) {
  // First we have to tell Rcpp what class to use for big.matrix objects.
  // This object stores the attributes of the big.matrix object passed to it
  // by R.
  XPtr<BigMatrix> xpMat(pBigMat);

  // To access values in the big.matrix, we need to create a MatrixAccessor
  // object of the appropriate type. Note that in every case we are still
  // returning a NumericVector: this is because big.matrix objects only store
  // numeric values in R, even if their type is set to 'char'. The types
  // simply correspond to the number of bytes used for each element.
  switch(xpMat->matrix_type()) {
  case 1:
    return Rcpp::wrap(BigColSums2<char>(xpMat));
  case 2:
    return Rcpp::wrap(BigColSums2<short>(xpMat));
  case 4:
    return Rcpp::wrap(BigColSums2<int>(xpMat));
  case 8:
    return Rcpp::wrap(BigColSums2<double>(xpMat));
  // case 1:
  //   return BigColSums<char>(xpMat);
  // case 2:
  //   return BigColSums<short>(xpMat);
  // case 4:
  //   return BigColSums<int>(xpMat);
  // case 8:
  //   return BigColSums<double>(xpMat);
  default:
    // This case should never be encountered unless the implementation of
    // big.matrix changes, but is necessary to implement shut up compiler
    // warnings.
    throw Rcpp::exception("unknown type detected for big.matrix object!");
  }
}

[YaohuiZeng]

BTW, I removed the header lines in utilities.cpp as you pointed out. Sorry for the confusion. As you know, this file was previously automatically generated, but later I decided not to have RcppExports and just made manually changes in this file.

[privefl]

Last guess for me.
Do the functions in utilities.cpp need dispatchers too?

[YaohuiZeng]

I don't think so. See here https://github.com/aloysius-lim/bigrf/blob/master/src/growtree.cpp