nolanlab/rclusterpp

Memory efficient clustering in R for large datasets

rclusterpp's Issues

Dealing with compile warnings

@bnaras Taking a quick look at the warnings you shared, seems like most can be suppressed with some explicit suppression calls - created a new branch with those added: 6a17290

However, I'm not sure this would fly for CRAN submission. Do you have a sense as to whether or not this approach would be acceptable?

Possible memory leak

From Brian Ripley:

"But there is still the allocation problem shown by AddressSanitizer (see Writing R Extensions)

Name: Rclusterpp.hclust

Title: Hierarchical Clustering

Aliases: Rclusterpp.hclust

** Examples

h <- Rclusterpp.hclust(USArrests, method="ward", distance="euclidean")

==17367== ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x0000033e5560

#0 0x7f9f9a7f9aca in operator delete(void_) asan_rtl
#1 0x7f9f9d786dee in _gnu_cxx::new_allocator::deallocate(int, unsigned long) /usr/local/gcc48x/include/c++/4.8.2/ext/new_allocator.h:110
#2 0x47af48 in do_dotcall /data/gannet/ripley/R/svn/R-devel/src/main/dotcode.c:597

which seems to indicate a problem when leaving."

Rclusterpp.h file not found

devtools::install_github("nolanlab/Rclusterpp")

Downloading GitHub repo nolanlab/Rclusterpp@master
from URL https://api.github.com/repos/nolanlab/Rclusterpp/zipball/master
Installing Rclusterpp
'/Library/Frameworks/R.framework/Resources/bin/R' --no-site-file --no-environ --no-save --no-restore --quiet CMD INSTALL  \
  '/private/var/folders/x3/58m75cp15bl_6jkpclyt_6qc0000gn/T/RtmpoXJ6kN/devtoolse585c7f2673/nolanlab-Rclusterpp-740cb0b'  \
  --library='/Library/Frameworks/R.framework/Versions/3.4/Resources/library' --install-tests 

* installing *source* package ‘Rclusterpp’ ...
** libs
/usr/local/clang4/bin/clang++ -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I"/Library/Frameworks/R.framework/Versions/3.4/Resources/library/Rcpp/include" -I"/Library/Frameworks/R.framework/Versions/3.4/Resources/library/RcppEigen/include" -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c hclust.cpp -o hclust.o
hclust.cpp:7:10: fatal error: 'Rclusterpp.h' file not found
#include <Rclusterpp.h>
         ^~~~~~~~~~~~~~
1 error generated.
make: *** [hclust.o] Error 1
ERROR: compilation failed for package ‘Rclusterpp’
* removing ‘/Library/Frameworks/R.framework/Versions/3.4/Resources/library/Rclusterpp’
Installation failed: Command failed (1)

cant install Rclusterpp

Hi,

To install FLOWMAPR, i need to install Rclusterpp: install_github("nolanlab/Rclusterpp")

Please see session info below and the error message I get from Rstudio. Can anyone please help?

(I have both flowCore and devtools libraries loaded)

Best,
Stein-Erik

######################

R version 3.5.3 (2019-03-11) -- "Great Truth"
Copyright (C) 2019 The R Foundation for Statistical Computing
Platform: x86_64-apple-darwin15.6.0 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

[Workspace loaded from ~/Dropbox/Rstudio/FLOWMAPR/.RData]

library("flowCore")
library(devtools)
install_github("nolanlab/Rclusterpp")
Downloading GitHub repo nolanlab/Rclusterpp@master
✔ checking for file ‘/private/var/folders/_8/fpg279xx6lg2r8dhph4jpl0d91tsw8/T/RtmpZUdHXN/remotesa0db510b1c2b/nolanlab-Rclusterpp-740cb0b/DESCRIPTION’ ...
─ preparing ‘Rclusterpp’:
✔ checking DESCRIPTION meta-information ...
─ cleaning src
─ running ‘cleanup’
─ checking for LF line-endings in source and make files and shell scripts
─ checking for empty or unneeded directories
─ building ‘Rclusterpp_0.2.4.tar.gz’
Warning: invalid uid value replaced by that for user 'nobody'
Warning: invalid gid value replaced by that for user 'nobody'

Installing package into ‘/Users/mmasg/Library/R/3.5/library’
(as ‘lib’ is unspecified)

• installing source package ‘Rclusterpp’ ...
  ** libs
  /opt/local/bin/g++-mp-4.8 -arch x86_64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I"/Users/mmasg/Library/R/3.5/library/Rcpp/include" -I"/Users/mmasg/Library/R/3.5/library/RcppEigen/include" -I/usr/local/include -I../inst/include -fopenmp -fPIC -Wall -g -O2 -c hclust.cpp -o hclust.o
  In file included from /opt/local/include/gcc48/c++/bits/postypes.h:40:0,
  from /opt/local/include/gcc48/c++/bits/char_traits.h:40,
  from /opt/local/include/gcc48/c++/string:40,
  from /opt/local/include/gcc48/c++/stdexcept:39,
  from hclust.cpp:1:
  /opt/local/include/gcc48/c++/cwchar:44:19: fatal error: wchar.h: No such file or directory
  #include <wchar.h>
  ^
  compilation terminated.
  make: *** [hclust.o] Error 1
  ERROR: compilation failed for package ‘Rclusterpp’
• removing ‘/Users/mmasg/Library/R/3.5/library/Rclusterpp’
  Error in i.p(...) :
  (converted from warning) installation of package ‘/var/folders/_8/fpg279xx6lg2r8dhph4jpl0d91tsw8/T//RtmpZUdHXN/filea0db1eb74d3b/Rclusterpp_0.2.4.tar.gz’ had non-zero exit status

Cant install nolanlab/Rclusterpp

I recently updated R to the 3.6.1 version. I am interested in installing the nolanlab/Rclusterpp package in RStudio, but I receive the following error: " Could not find tools necessary to compile a package". . I tried installing the packages on their own (eg. nolanlab and Rclusterpp), but I receive the error message saying that the packages are not available (for R version 3.6.1). Any help would be greatly appreciated

No more on CRAN

Hi Robert,
The package is no more on CRAN:
https://cran.r-project.org/web/packages/Rclusterpp/index.html
I think you should update the installation process.
Many thanks for maintaining this package.
Best.

/bin/sh: g++-mp-4.8: command not found

Hi there,

I'm trying to install Rclusterpp for R on my os x laptop. I have followed the installation instructions here: https://github.com/nolanlab/Rclusterpp/wiki/Getting-Started and when I try to run install.packages(c("Rcpp","RcppEigen","Rclusterpp"),type="source") in R, I keep getting the error /bin/sh: g++-mp-4.8: command not found. Any ideas on how to get around this?

Thanks,
Simon

occasional segfault with Rclusterpp.hclust

I'm running into segfault errors occasionally (maybe 1/10 times) when running Rclusterpp.hclust. Any idea why this might be occurring?

I've uploaded a simple (simulated) data set where I run into a segfault as a gist for reference.
This data set only appears to cause a segfault with method = "ward", but I've also observed segfaults with other linkages (e.g. method = "complete") as well (for other data).

---

Printing out the sessionInfo.

 library(Rclusterpp)
 # Loading required package: Rcpp
 # Loading required package: RcppEigen
 
 sessionInfo()
 # R version 3.4.2 (2017-09-28)
 # Platform: x86_64-apple-darwin15.6.0 (64-bit)
 # Running under: OS X El Capitan 10.11.6
 # 
 # Matrix products: default
 # BLAS/LAPACK: /usr/local/Cellar/openblas/0.2.20/lib/libopenblasp-r0.2.20.dylib
 # 
 # locale:
 # [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
 # 
 # attached base packages:
 # [1] stats     graphics  grDevices utils     datasets  methods   base
 # 
 # other attached packages:
 # [1] Rclusterpp_0.2.3    RcppEigen_0.3.3.3.1 Rcpp_0.12.16
 # 
 # loaded via a namespace (and not attached):
 # [1] compiler_3.4.2  Matrix_1.2-12   tools_3.4.2     grid_3.4.2
 # [5] lattice_0.20-35

The data can be loaded from the gist.

 dat <- read.table(paste0("https://gist.githubusercontent.com/pkimes/",
                          "9d84b603a3f856b100c33e67c7c477fd/raw/",
                          "079ab207f980edafbb893562c6f4ad9e9929338c/",
                          "testdata.txt"))

The standard stats::hclust appears fine.

 hclust(dist(dat), method = "ward.D2")
 # 
 # Call:
 # hclust(d = dist(dat), method = "ward.D2")
 # 
 # Cluster method   : ward.D2
 # Distance         : euclidean
 # Number of objects: 48

Unfortunately, I run into a segfault with Rclusterpp.hclust.

 Rclusterpp.hclust(dat)
 # 
 #  *** caught segfault ***
 # address 0x5, cause 'memory not mapped'
 # 
 # Traceback:
 #  1: .Call("hclust_from_data", data = x, link = as.integer(method),     dist = as.integer(distance), p = as.numeric(p), DUP = FALSE,     NAOK = FALSE, PACKAGE = "Rclusterpp")
 #  2: Rclusterpp.hclust(dat)
 # 
 # Possible actions:
 # 1: abort (with core dump, if enabled)
 # 2: normal R exit
 # 3: exit R without saving workspace
 # 4: exit R saving workspace

Odd results from Rclusterpp.hclust()

I started using Rclusterpp.hclust(), but have found that it gives odd / incorrect results with some data sets that I've used it on. Of note, the returned object is not the same as that of hclust() {identical(h.fit$merge, r.fit$merge) && all.equal(h.fit$height, r.fit$height)} returns FALSE for several distances and methods. Even more strangely when I cut the resulting object with cutree I get more or less segments than requested (e.g. table(cutree(r.fit, k = 2)) shows I get WAY more than 2 segment solutions).

Below is a link to an .Rdata file with 4 matrices (mydata1, mydata2, mydata3, mydata4) that all exhibit these problems with Rclusterpp.hclust().

https://app.box.com/s/8gmv7rsqoetl8owchd3bi9j33q735jzy

The problems I'm seeing are produced using the below code, although I also experience issues with other distances and methods available in Rclusterpp.hclust():

h.fit <- hclust(d = dist(mydata4, method = "manhattan"), method = "complete")
r.fit <- Rclusterpp.hclust(mydata4, distance = "manhattan", method = "complete")

identical(h.fit$merge, r.fit$merge) && all.equal(h.fit$height, r.fit$height)

apply(cutree(h.fit, k = 2:10), 2, table)
apply(cutree(r.fit, k = 2:10), 2, table)

My R session info is as follows:
R version 3.3.1 (2016-06-21)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1

locale:
[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C LC_TIME=English_United States.1252

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] Rclusterpp_0.2.3 RcppEigen_0.3.2.9.0 Rcpp_0.12.8

loaded via a namespace (and not attached):
[1] Matrix_1.2-7.1 tools_3.3.1 grid_3.3.1 lattice_0.20-34

CRAN checks failed

@mlinderm You submitted this on CRAN , I approved, and it did not pass pretest checks. See message below. It passes on Windows, however, without that warning.

using log directory ‘/srv/hornik/tmp/CRAN/Rclusterpp.Rcheck’
* using R Under development (unstable) (2022-08-11 r82712)
* using platform: x86_64-pc-linux-gnu (64-bit)
* using session charset: UTF-8
* checking for file ‘Rclusterpp/DESCRIPTION’ ... OK
* checking extension type ... Package
* this is package ‘Rclusterpp’ version ‘0.2.6’
* package encoding: UTF-8
* checking CRAN incoming feasibility ... NOTE
Maintainer: ‘Balasubramanian Narasimhan <[email protected]>’

New submission

Package was archived on CRAN

Possibly misspelled words in DESCRIPTION:
  Linkable (3:8)

CRAN repository db overrides:
  X-CRAN-Comment: Archived on 2018-05-09 as check problems were not
    corrected despite reminders.
* checking package namespace information ... OK
* checking package dependencies ... OK
* checking if this is a source package ... OK
* checking if there is a namespace ... OK
* checking for executable files ... OK
* checking for hidden files and directories ... OK
* checking for portable file names ... OK
* checking for sufficient/correct file permissions ... OK
* checking serialization versions ... OK
* checking whether package ‘Rclusterpp’ can be installed ... [10s/10s] WARNING
Found the following significant warnings:
  ../inst/include/Rclusterpp/util.h:9:51: warning: ‘template<class _Arg1, class _Arg2, class _Result> struct std::binary_function’ is deprecated [-Wdeprecated-declarations]
  ../inst/include/Rclusterpp/method.h:64:64: warning: ‘template<class _Arg1, class _Arg2, class _Result> struct std::binary_function’ is deprecated [-Wdeprecated-declarations]
See ‘/srv/hornik/tmp/CRAN/Rclusterpp.Rcheck/00install.out’ for details.
* checking package directory ... OK
* checking for future file timestamps ... OK
* checking ‘build’ directory ... OK
* checking DESCRIPTION meta-information ... OK
* checking top-level files ... OK
* checking for left-over files ... OK
* checking index information ... OK
* checking package subdirectories ... OK
* checking R files for non-ASCII characters ... OK
* checking R files for syntax errors ... OK
* checking whether the package can be loaded ... OK
* checking whether the package can be loaded with stated dependencies ... OK
* checking whether the package can be unloaded cleanly ... OK
* checking whether the namespace can be loaded with stated dependencies ... OK
* checking whether the namespace can be unloaded cleanly ... OK
* checking loading without being on the library search path ... OK
* checking use of S3 registration ... OK
* checking dependencies in R code ... OK
* checking S3 generic/method consistency ... OK
* checking replacement functions ... OK
* checking foreign function calls ... OK
* checking R code for possible problems ... [2s/2s] OK
* checking Rd files ... [0s/0s] OK
* checking Rd metadata ... OK
* checking Rd line widths ... OK
* checking Rd cross-references ... OK
* checking for missing documentation entries ... OK
* checking for code/documentation mismatches ... OK
* checking Rd \usage sections ... OK
* checking Rd contents ... OK
* checking for unstated dependencies in examples ... OK
* checking line endings in shell scripts ... OK
* checking line endings in C/C++/Fortran sources/headers ... OK
* checking line endings in Makefiles ... OK
* checking compilation flags in Makevars ... OK
* checking for GNU extensions in Makefiles ... OK
* checking for portable use of $(BLAS_LIBS) and $(LAPACK_LIBS) ... OK
* checking use of PKG_*FLAGS in Makefiles ... OK
* checking use of SHLIB_OPENMP_*FLAGS in Makefiles ... OK
* checking include directives in Makefiles ... OK
* checking pragmas in C/C++ headers and code ... OK
* checking compilation flags used ... OK
* checking compiled code ... OK
* checking installed files from ‘inst/doc’ ... OK
* checking files in ‘vignettes’ ... OK
* checking examples ... [0s/0s] OK
* checking for unstated dependencies in ‘tests’ ... OK
* checking tests ... [1s/0s] OK
  Running ‘doRUnit.R’ [0s/0s]
* checking for unstated dependencies in vignettes ... OK
* checking package vignettes in ‘inst/doc’ ... OK
* checking re-building of vignette outputs ... [1s/1s] OK
* checking PDF version of manual ... OK
* checking HTML version of manual ... OK
* checking for non-standard things in the check directory ... OK
* checking for detritus in the temp directory ... OK
* DONE
Status: 1 WARNING, 1 NOTE

'Rclusterpp' is not available (for R version 3.5.1)

install.packages("Rclusterpp")
--- Please select a CRAN mirror for use in this session ---
Warning message:
package 'Rclusterpp' is not available (for R version 3.5.1)
library(devtools)
devtools::install_github("nolanlab/spade")
Downloading GitHub repo nolanlab/spade@master
tar: Failed to set default locale
tar: Failed to set default localeSkipping 1 packages not available: Rclusterpp
Installing 15 packages: BH, Biobase, BiocGenerics, DEoptimR, Rclusterpp, corpcor, flowCore, graph, igraph, matrixStats, mvtnorm, pcaPP, pkgconfig, robustbase, rrcov
Error: (converted from warning) package 'Rclusterpp' is not available (for R version 3.5.1)

Package no longer on CRAN

FYI, this is the information I receive from CRAN

Package ‘Rclusterpp’ was removed from the CRAN repository.

Formerly available versions can be obtained from the archive.

Archived on 2018-05-09 as check problems were not corrected despite reminders.

To install, should I use the last archived version or do you support installation directly from github via devtools::install_github()?

Thanks, Brandon