sofievg/flowsom

Using self-organizing maps for visualization and interpretation of cytometry data

flowsom's Issues

Exporting MapDataToCodes function

Hi Sofie,

I am going to use MapDataToCodes to do an "upscaling" in order to assign cluster id to cells that have been left out during the downsampling when analyzing large sets. If I understand correctly the aim of this function, it maps cells to codes using the 1st nearest neighbor. Do you plan to export this function ? This could be useful in your pipeline when dealing with large datasets. Of course, there are many alternatives to achieve this goal but MapDataToCodes is a correct one. It probably could be parallelized as well.

If you think about exporting it, what about adding some checks before calling C code. I forgot to add colnames and crashed my R session :-(

FlowSOM/R/2_buildSOM.R

Line 202 in 7c78eb1

as.double(newdata[,colnames(codes)]),

Here are some checks I think about, but absolutely untested.

if (is.null(colnames(codes)) stop("Columns of codes must have names.")
if (is.null(colnames(newdata)) stop("Columns of newdata must have names.")
if (setdiff(colnames(codes), colnames(newdata)) stop("Colnames of codes must be present in newdata.")

Let me know,
Samuel

TestOutliers

Hi,
I also looked at TestOutliers function because it is just below the NewData function. I am not sure whether an absolute limit to call a value an outlier fits all the needs, especially when one population might be very abundant or very rare. I imagine the current threshold of 4 MAD will rarely raise an alert. You probably thought about adjusting the threshold according to the number of events. If so could tell for which reason you didn't retain this approach?
As this view sounds familiar to my mind, maybe we already discussed about it. Sorry if this is the case.
Best,
Samuel

FlowSOM populations not exported into FlowJo

Hi! I'm having an issue trying to use FlowSOM in FlowJo. Basically I don't get any population after applying the plugin. The R path looks fine, but other that a CSV file I don't get anything else. I'm attaching an image with the result in FlowJo. Can somebody help? Thanks!!!

I want to present some fancy data next week! ;-)

view is not checked

Hi Sophie,
I wrote tsne instead of tSNE, and I got an error
Error in igraph::plot.igraph(fsom$MST$g, vertex.shape = "star", vertex.label = NA, : object 'lty' not found
It seems that view is not controlled.
Best.

FlowSOM/R/3_buildMST.R

Line 1101 in 50daac2

view="MST", #"grid","tSNE"

FlowSOM/R/3_buildMST.R

Lines 1152 to 1159 in 50daac2

	switch(view,
	MST = { layout <- fsom$MST$l
	lty <- 1},
	grid = { layout <- as.matrix(fsom$map$grid)
	lty <- 0},
	tSNE = { layout <- fsom$MST$l2
	lty <- 0}
	)

PlotStars met: MST=2, plot grid option

When using PlotStars of FlowSOM and using MST=2, what does each grid of 10x10 represent? Why, even if the number of markers change, the grid size stays the same. Could you maybe give some reading on how exactly the tree is build & maybe some reading about the algorithm?
Thank you

ReadInput/AddFlowframe: doc

Hi. To be more precise, when no compensation matrix is given, but compensation is wanted (compensate = TRUE), the spillover matrix is searched in the first flowFrame (or FCS file), and the next flowFrames (or FCS files) will be compensated with this spillover matrix, not their own.
To be short, the compensation matrix of the flowSOM object is the one of the first FCS, and all added FCS are compensated with it.

FlowSOM/R/1_readInput.R

Lines 10 to 13 in 7d1fcfc

	#' @param compensate logical, does the data need to be compensated
	#' @param spillover spillover matrix to compensate with
	#' If \code{NULL} and compensate=\code{TRUE}, we will
	#' look for \code{$SPILL} description in fcs file.

FlowSOM/R/1_readInput.R

Lines 130 to 163 in 7d1fcfc

	#' Add a flowFrame to the data variable of the FlowSOM object
	#'
	#' @param fsom FlowSOM object, as constructed by the ReadInput function
	#' @param flowFrame flowFrame to add to the FlowSOM object
	#'
	#' @return FlowSOM object with data added
	#'
	#' @seealso \code{\link{ReadInput}}
	AddFlowFrame <- function(fsom, flowFrame){
	# Compensation
	if(fsom$compensate){
	if(is.null(fsom$spillover)){
	if(!is.null(flowFrame@description$SPILL)){
	fsom$spillover <- flowFrame@description$SPILL
	} else if (!is.null(flowFrame@description$`$SPILLOVER`)){
	if(class(flowFrame@description$`$SPILLOVER`)=="matrix"){
	fsom$spillover = flowFrame@description$`$SPILLOVER`
	flowFrame@description$SPILL = fsom$spillover
	} else {
	spilloverStr <- strsplit(
	flowFrame@description$`$SPILLOVER`,
	",")[[1]]
	n <- as.numeric(spilloverStr[1])
	fsom$spillover <- t(matrix(as.numeric(spilloverStr[(n+2):
	length(spilloverStr)]),ncol=n))
	colnames(fsom$spillover) <- spilloverStr[2:(n+1)]
	flowFrame@description$SPILL <- fsom$spillover
	}
	} else {
	stop("No compensation matrix found")
	}
	}
	flowFrame <- flowCore::compensate(flowFrame, fsom$spillover)
	}

L135 ... and spillover matrix instantiated (?)
Thanks for reviewing my comments. Best.

Calling FlowSOM:::MapDataToCodes produces a segmentation fault

I'm attempting to use a weights matrix to assign cluster labels to a new dataset, but when I do so, I get this error:

 *** caught segfault ***
address 0x7fd2240f5ea8, cause 'memory not mapped'

Traceback:
 1: FlowSOM:::MapDataToCodes(somWeights, fovPixelData[, markers])

somWeights contains a 10x10 matrix computed by SOM. fovPixelData also contains a numeric matrix of about 1M rows.

Some more information:

• fovPixelData is initially read in as a tibble from a .feather file using arrow::read_feather. It needs to be stored this way for Python data frame interoperability. The same segfaults come up whether or not I coerce fovPixelData to a matrix or not
• somWeights is read in from an intermediate HDF5 file using h5read from rhdf5.

GetFlowJoLabels error: FCS files not found

Hi!
I am working with some flow cytometry data. We initially employed a cleaning strategy in Flow Jo by using FlowAI. We exported the good events into new FCS files, made the gating strategy in a FlowJo workspace and applied a R script to build the FlowSOM analysis, which worked quite well. However, we decided to re-build the analysis using only the CD4+ population. We exported the CD4+ population (from the "good events" exported FCS files) into new FCS files, but when we try to use the GetFlowJoLabels command it returns with the following error:

manual <- GetFlowJoLabels(list_fcs, gating_wsp, cell_types = cell_types)
[1] "Processing export_t panel 02_A2_Viable_CD4+.fcs"
Error in GetFlowJoLabels(list_fcs, gating_wsp, cell_types = cell_types) :
File not found. Files available:
export_t panel 02_A2_Viable_CD4+.fcs
export_t panel 03_A2_Viable_CD4+.fcs
export_t panel 04_A2_Viable_CD4+.fcs
...

As you see, it reports that the file is not found, but the files available has exactly the same name. We double checked that names in the "list_fcs", "gating_wsp" and the fcs files has the same name.

FlowSOM functions cannot be called directly by user

Hi,

I having been working through a FlowSOM workflow and I found that a few of the documented functions cannot be called by the user. I checked the FlowSOM Namespace file on Github and found that the functions such as AddFlowFrame, computeBackgroundColor, Dist.MST, and MapDataToCodes are not exported in the Namespace file. This prevents R from finding these functions even if the package is loaded. I would like to be able to use the MapDataToCodes so that I can map new sample data onto an existing plot. Below is the link to an article that I found to be helpful while trying to identify the problem.

http://alyssafrazee.com/2014/01/21/namespaces.html

I would greatly appreciate any help.
Thank you!

Installation of FlowSOM failed

Hi,

Installation of FlowSOM failed with a following message:

Error: Failed to install 'FlowSOM' from GitHub:
  System command error, exit status: 1, stdout + stderr (last 10 lines):
E> installing to /private/var/folders/wr/dljlk_z934d83zpz6r0fx3n1l1pf6f/T/RtmpjYWCJx/Rinstf161704759c1/00LOCK-FlowSOM/00new/FlowSOM/libs
E> ** R
E> ** inst
E> ** byte-compile and prepare package for lazy loading
E> Error: object ‘getIndiceMat’ is not exported by 'namespace:flowWorkspace'
E> Execution halted
E> ERROR: lazy loading failed for package ‘FlowSOM’
E> * removing ‘/private/var/folders/wr/dljlk_z934d83zpz6r0fx3n1l1pf6f/T/RtmpjYWCJx/Rinstf161704759c1/FlowSOM’
E>       -----------------------------------
E> ERROR: package installation failed

A culprit is object ‘getIndiceMat’ is not exported by 'namespace:flowWorkspace'. flowWorkspace is installed as follows;

packageVersion("flowWorkspace")
[1] ‘3.33.9’

Here is sessionInfo;

sessionInfo()
R version 3.6.1 (2019-07-05)
Platform: x86_64-apple-darwin18.6.0 (64-bit)
Running under: macOS Mojave 10.14.6

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /usr/local/Cellar/openblas/0.3.7/lib/libopenblasp-r0.3.7.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] flowWorkspace_3.33.9 biomaRt_2.41.8       Rsubread_1.99.3      edgeR_3.27.13        limma_3.41.16       
 [6] fastcluster_1.1.25   TeachingDemos_2.10   ggpubr_0.2.3         magrittr_1.5         ggplot2_3.2.1       
[11] data.table_1.12.2    stringr_1.4.0        BiocManager_1.30.4   devtools_2.2.1       usethis_1.5.1       

loaded via a namespace (and not attached):
 [1] ncdfFlow_2.31.4      matrixStats_0.55.0   fs_1.3.1             bit64_0.9-7          RColorBrewer_1.1-2  
 [6] progress_1.2.2       httr_1.4.1           rprojroot_1.3-2      Rgraphviz_2.29.0     tools_3.6.1         
[11] backports_1.1.4      R6_2.4.0             DT_0.9               KernSmooth_2.23-15   DBI_1.0.0           
[16] lazyeval_0.2.2       BiocGenerics_0.31.6  colorspace_1.4-1     withr_2.1.2          gridExtra_2.3       
[21] tidyselect_0.2.5     prettyunits_1.0.2    processx_3.4.1       bit_1.1-14           curl_4.2            
[26] compiler_3.6.1       graph_1.63.0         cli_1.1.0            Biobase_2.45.1       desc_1.2.0          
[31] scales_1.0.0         hexbin_1.27.3        callr_3.3.2          askpass_1.1          rappdirs_0.3.1      
[36] digest_0.6.21        pkgconfig_2.0.3      htmltools_0.3.6      sessioninfo_1.1.1    dbplyr_1.4.2        
[41] htmlwidgets_1.3      rlang_0.4.0          rstudioapi_0.10      flowCore_1.51.7      RSQLite_2.1.2       
[46] dplyr_0.8.3          Rcpp_1.0.2           munsell_0.5.0        S4Vectors_0.23.23    stringi_1.4.3       
[51] MASS_7.3-51.4        zlibbioc_1.31.0      pkgbuild_1.0.5       BiocFileCache_1.9.1  grid_3.6.1          
[56] blob_1.2.0           parallel_3.6.1       crayon_1.3.4         lattice_0.20-38      hms_0.5.1           
[61] locfit_1.5-9.1       zeallot_0.1.0        ps_1.3.0             pillar_1.4.2         ggsignif_0.6.0      
[66] stats4_3.6.1         pkgload_1.0.2        XML_3.98-1.20        glue_1.3.1           latticeExtra_0.6-28 
[71] remotes_2.1.0        RcppParallel_4.4.3   vctrs_0.2.0          testthat_2.2.1       gtable_0.3.0        
[76] openssl_1.4.1        purrr_0.3.2          assertthat_0.2.1     IDPmisc_1.1.19       tibble_2.1.3        
[81] flowViz_1.49.2       AnnotationDbi_1.47.1 memoise_1.1.0        IRanges_2.19.16      ellipsis_0.3.0

and my OSX version;

sw_vers
ProductName:	Mac OS X
ProductVersion:	10.14.6
BuildVersion:	18G95

Any help will be appreciated.

Thanks.

Question som$map$codes

Hello,

I am a bit struggling to understand the details of FlowSOM. Imagine that you have a flow set that you have clustered.
som <- BuildSOM(fsom, colsToUse = some_colors )
cell_clustering_som <- som$map$mapping[,1]

So, cell_clustering_som tell us to which cluster a given line of the flow set belongs.
If I have:
expr_mat=fsApply(fsom,exprs)

What is the relation between expr_mat and som$map$codes?
I was thinking that "codes" represent the average expression for a given cluster, so if I do (for instance for cluster 1):
apply(expr_mat[cell_clustering_som==1, some_colors],2,mean))
I should get the "codes" for cluster 1, but I don't.

Simply, is there a way to get som$map$codes from cell_clustering_som and expr_mat?

Thank you for any answer.

FlowSOM fails after CytoNorm

Hi Sofie,
Thank-you for FlowSOM!
I was able to successfully run FlowSOM on the raw data but failed when the fcs files were normalized using CytoNorm. I believe, this error is because of introductions of NAs. How can I debug / circumvent this?

     sce <- cluster(sce, features = "type",
     +                xdim = 10, ydim = 10, maxK = 10,
     +                verbose = T, seed = 1)
     > o running FlowSOM clustering...
     > Error in SOM(fsom$data[, colsToUse], silent = silent, ...) : 
     > NA/NaN/Inf in foreign function call (arg 1)

By default,

     > options("na.action")
     $na.action
     [1] "na.omit"

am using R v4.0.0
and FlowSOM v1.20.0

Thanks a lot!
Vivek

Calling MapDataDataToCodes produces long vectors error

Hello,

I am running FlowSOM:::MapDataToCodes(someWeights, as.matrix(fovPixelData)) which produces the following error:

Error in FlowSOM:::MapDataToCodes(someWeights, as.matrix(fovPixelData)) : 
  long vectors (argument 1) are not supported in .C

someWeights is a 100 x 10 numerical matrix, and fovPixel data is a large file (297923080 obs. of 10 variables). fovPixel is produced by reading a .feather file as data.table(arrow::read_feather(file))

Do you have any thoughts on what is causing this error?

Thanks!

CountGroups Error

Tried running this:

groups <- CountGroups(fSOM_all, data_names, plot = TRUE, silent = FALSE)

and got this error:

Error in tapply(x, INDEX = factor(rep(names(groups), lapply(groups, length)),  : 
  arguments must have same length

traceback()
5: stop("arguments must have same length")
4: tapply(x, INDEX = factor(rep(names(groups), lapply(groups, length)), 
       levels = names(groups)), mean)
3: FUN(newX[, i], ...)
2: apply(pctgs, 2, function(x) {
       tapply(x, INDEX = factor(rep(names(groups), lapply(groups, 
           length)), levels = names(groups)), mean)
   })
1: CountGroups(fSOM_all, data_names, plot = TRUE, silent = FALSE)

The plots are produced but when trying to do:

PlotGroups(fSOM_all, groups, view = "MST")

I get:

Error in groups$means : object of type 'closure' is not subsettable

Does the error message imply that the flowframes supplied to ReadInput have to have equal number of events?

FlowSOM installation issues with boost library

I'm trying to download FlowSOM using the BiocManager::install('FlowSOM') but am running into issues with the following error:

g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -DROUT -I../inst/include -DBOOST_NO_AUTO_PTR -I'/usr/local/lib/R/site-library/Rcpp/include' -I'/usr/local/lib/R/site-library/BH/include' -I'/usr/local/lib/R/site-library/RProtoBufLib/include' -I'/usr/local/lib/R/site-library/Rhdf5lib/include' -I'/usr/local/lib/R/site-library/RcppArmadillo/include' -I'/usr/local/lib/R/site-library/RcppParallel/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-XqSJAD/r-base-4.0.4=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c boost/libs/filesystem/src/operations.cpp -o boost/libs/filesystem/src/operations.o
boost/libs/filesystem/src/operations.cpp: In function ‘boost::filesystem::path boost::filesystem::detail::relative(const boost::filesystem::path&, const boost::filesystem::path&, boost::system::error_code*)’:
boost/libs/filesystem/src/operations.cpp:2421:46: error: no matching function for call to ‘weakly_canonical(const boost::filesystem::path&, boost::system::error_code*)’
 2421 |   path wc_base(weakly_canonical(base, &tmp_ec));

I have g++ version 10.2.1 on Debian. Let me know if there's an issue with my g++ version or the way I'm installing FlowSOM.

UPDATE: seems to be an issue with the cytolib library in particular.

Porting FlowSOM to python

Hi @SofieVG,
We are thinking about porting FlowSOM to python so that it can integrate more seamlessly into our pipeline. We would make the resulting repo open source, but we just wanted to check in with you before starting to make sure that's okay. There's currently not a license on the repo.

Thanks,
Noah

clarification of BuildSOM colsToUse

Hi, I understand my question might be too simple, however, i have been trying to plot & experiment with different colsToUse parameters & still couldn't understand what it specifies.
when using the BuildSOM function of FlowSOM, the parameter colsToUse says it specifies column names or indices to use for building the SOM. However, in the tutorial the parameter is specified as colsToUse=c(9,12,14:18). Does this mean that the self-organizing map will be build using the columns 9 & 12, for markers 14 through 18?
ex: fSOM=BuildSOM(fSOM,colsToUse=c(9,12,14:18))

Thank you

Speedup when threshold < 1

Hi,
I read the following code and I am wondering about the following proposal.

FlowSOM/src/som.c

Lines 120 to 141 in 1e5650e

	nearest = 0;
	/* calculate distances in x and y spaces, and keep track of the
	nearest code */
	for (cd = 0; cd < ncodes; cd++) {
	xdists[cd] = distf(&data[i], &codes[cd], px, n, ncodes);
	if (xdists[cd] < xdists[nearest]) nearest = cd;
	}
	if (threshold < 1.0) threshold = 0.5;
	alpha = alphas[0] - (alphas[0] - alphas[1]) * (double)k/(double)niter;
	for (cd = 0; cd < ncodes; cd++) {
	if(nhbrdist[cd + ncodes*nearest] > threshold) continue;
	for(j = 0; j < px; j++) {
	tmp = data[i + j*n] - codes[cd + j*ncodes];
	change += fabs(tmp);
	codes[cd + j*ncodes] += tmp * alpha;
	}
	}
	threshold -= thresholdStep;

When threshold reaches 1, only the best matching unit is updated. The loop L131-L139 looking for the nearest units below the threshold is useless. Only the calculation of the inner loop L134-L138 is needed with cd being nearest. The code at L128 is a weird shortcut IMHO. With the default parameters (grid xdim = ydim = 10, radius is 6), the gain applies in the last 2 loops out of 10 (rlen = 10).
Let me know if there is something I misunderstood.

Is the integer cast in abs() really intended?

For various reasons, I am writing a C++ implementation of som.c, and I noticed

FlowSOM/src/som.c

Lines 133 to 134 in 6f79b8b

	tmp = data[i + j*n] - codes[cd + j*ncodes];
	change += abs(tmp);

tmp is double-precision but abs() takes integer arguments. This call will truncate tmp to integer before actually computing the absolute difference. Nothing will be added to change if all tmp lie in (-1, 1). As it stands now, the iterations will terminate if all tmp have absolute values less than 1. Is this intentional? Did you mean to use fabs() instead?

Regardless, I don't understand the motivation behind the stop condition of change < 1. A threshold of 1 seems pretty arbitrary when the scale of the input data could be anything. Why not just terminate when the change in the code coordinates drops below a certain level (e.g., defined as some small fraction of the standard deviation across codes)?

Recent error installing flowsom from cran

https://stackoverflow.com/questions/45318188/getting-error-in-function-igraph-write-graph-graphml-while-installing-igrap/45362040#comment77805669_45362040

There is a new error from igraph which flowsom requires. This error will occur when installing flowsom from CRAN, so the workaround is to install directly from github.

install.packages("devtools")
library(devtools)
install_github("igraph/rigraph")

Error: “In function ‘igraph_write_graph_graphml’:” while installing igraph package in R

PlotStars from FlowSOMSubset

Dear Sofie,

Thanks for this excellent package!

I would like to PlotStars for an individual sample from the BuildMST object.

I've tried subsetting the BuildMST object using the FlowSOMSubset command:

fSOM2 <- FlowSOMSubset(fSOM,(fSOM$metaData[[1]][1]):(fSOM$metaData[[1]][2]))

And get the following error when attempting to pass fSOM2 to PlotStars...

Error in fsom$map$medianValues[, markers, drop = FALSE] :
no 'dimnames' attribute for array

For the unsubsetted BuildMST object, fsom$map$medianValues has markers for column names. The subsetted object has numbers instead. Maybe this is part of the issue....?

By adding back the correct column names to fsom$map$medianValues, the plot seems to work fine.

Perhaps I'm using your code incorrectly, or is this a bug that needs a small tweak?

Thanks very much

FlowCore package issue - I tried updating all, none or some (individually) Any idea?

package ‘flowCore’ successfully unpacked and MD5 sums checked
Warning: cannot remove prior installation of package ‘flowCore’

The downloaded binary packages are in
C:\Users\priya\AppData\Local\Temp\Rtmp6bA2YF\downloaded_packages
installation path not writeable, unable to update packages: cluster, foreign, MASS, Matrix, nlme, survival
Old packages: 'bit', 'data.table', 'purrr', 'rlang', 'stringi'

Error output in R from FLowSOM plugin:
suppressMessages(library("FlowSOM"))
Error: package or namespace load failed for 'FlowSOM' in loadNamespace(j <- i[[1L]], c(lib.loc, .libPaths()), versionCheck = vI[[j]]):
there is no package called 'flowCore'
Execution halted

Add seed for automatic meta cluster inference

Hi Sofie,
I just realised that there is no way to set a seed when FlowSOM is used to automatically infer the optimal number of meta clusters. I have tried passing the seed as seed parameter, but it didn't seem to use it at all. Having the ability to set seed is handy as then I can repeatedly rerun the automatic inference and get the same result. Otherwise, each time I run it, the meta clustering looks different.

I've looked through the code, and I think the change is relatively simple, just explicitly pass the seed parameter into MetaClustering, DetermineNumberOfClusters, and the consensus function (see attached screenshot).

What do you think?

Save PlotStars command

Hello,

I am trying to save a "PlotStars" on a png file but to do that I need the command more than the GUI way. Is there a way to do that?

Query - FlowSOM

Hi Sofie,

Thank you for developing flowSOM.

I had a question on how to find the number of cells within a particular cluster in a flowSOM object?

Thank you,
Santosh

SOM should protect against inputs with no column names

In the following example:

library(FlowSOM)
set.seed(1000)
x <- matrix(runif(10000), ncol=10)
out <- SOM(x, xdim=5, ydim=5)
out$mapping[,1]
##   [1] 5 5 5 5 5 5 5 5 5 5 5 5 5 5 # and on it goes

All observations are assigned the same code - odd. Diving into the source reveals the offending line:

FlowSOM/R/2_buildSOM.R

Lines 201 to 209 in 6f79b8b

	nnCodes <- .C("C_mapDataToCodes",
	as.double(newdata[,colnames(codes)]),
	as.double(codes),
	as.integer(nrow(codes)),
	as.integer(nrow(newdata)),
	as.integer(ncol(codes)),
	nnCodes = integer(nrow(newdata)),
	nnDists = double(nrow(newdata)),
	distf = as.integer(distf))

Here, colnames(codes) is NULL, which causes data to be subsetted to an empty numeric vector. The C code subsequently performs out-of-bounds accesses, leading to very difficult memory-related bugs downstream of using SOM().

An expedient solution would be to simply enforce identical colnames in any non-NULL value of codes. If codes=NULL, then it's derived from the data and there is no need for a separate check that the columns are matching.

Building error on Bioconductor

Hello Sofie,

I tried to send the following email to you via the email listed in the DESCRIPTION file
but it was undeliverable. Please double check that this email is correct so that we may
be able to contact you in the future. It is crucial that the packages within Bioconductor
list an active maintainer email that responds to inquires.

The next Bioconductor release 3.16 is scheduled for November 2nd. Commits to the
current 3.15 release should be made by October 14th and the last day to commit
to the devel branch is October 28th. We like to make an effort to have all
Bioconductor packages building and checking without ERROR or TIMEOUT in both
release and devel versions.

Currently your package is producing an ERROR on the Windows builder in both
the release and devel versions of Bioconductor. Could you please investigate
the error as soon as possible?

Release version - https://bioconductor.org/checkResults/3.15/bioc-LATEST/FlowSOM/
Devel version - https://bioconductor.org/checkResults/3.16/bioc-LATEST/FlowSOM/

If you need any assistance please feel free to ask questions at
[email protected].

Thank you,
Kayla

FlowSOMSubset does not subset CV values

When I build a FlowSOM object on multiple files, then take a subset of it for one file using FlowSOMSubset, then call GetMFIs on that subset, the MFIs are returned just for the subset. But when I do the same thing using GetCVs, the values are unchanged.

Looking at the FlowSOMSubset code, I can see that the $map$medianValues get specifically updated, but I see no similar code for $map$cvValues

I think this is a bug, but I may be misunderstanding something.

Release version out of date compared to Flowjo Plugin.

Hello.

I am interested in running FlowSOM with custom code in R.
I currently use the plugin version for Flowjo.

The Flowjo version is up to 3.0.18, and was released recently.
The version available here is more than 5 years old.

Can I get a copy of the most recent release?

BuildSOM produces NaN/Infs error on the C part

Using different versions of FlowSOM from bioconda I get this strange behaviour where BuilSOM C part produces:

Building SOM

Error: NA/NaN/Inf in foreign function call (arg 1)

If I look at the traceback:

> traceback()
3: .C("C_SOM", data = as.double(data), codes = as.double(codes),
       nhbrdist = as.double(nhbrdist), alpha = as.double(alpha[[i]]),
       radius = as.double(radius[[i]]), xdists = double(nCodes),
       n = as.integer(nrow(data)), px = as.integer(ncol(data)),
       ncodes = as.integer(nCodes), rlen = as.integer(rlen), distf = as.integer(distf))
2: SOM(fsom$data[, colsToUse], silent = silent, ...)
1: BuildSOM(fs, colsToUse = columns, xdim = xgrid, ydim = ygrid)

I have tried different versions of flowsom, flowdensity and flowcore (within conda) in the steps that generate the merged fcs used as input for BuildSOM, with the same result. However, certain combination of versions installed directly over R does work. The direct R version does fail when I feed it the input file created with the conda flowsom/flowdensity/flowcore installations. Any ideas or anything I should be looking for?

Thanks!

Accessing MapDataFromCodes in FlowSOM/installation error using devtools

I'm trying to use FlowSOM to train on a subset of our data and apply the trained weights to cluster the entire dataset. The default settings of the FlowSOM algorithm, however, seem to only support training and clustering on the entire dataset with no intuitive way to allow for subsetting.

Looking at the source code and documentation, it looks like this can be done by passing the subsetted data into SOM, then passing the trained weights and the entire dataset into MapDataToCodes. However, it seems like we would need version 1.4.0 of the library to do so according to RDocumentation, and BiocManager only installs 1.19.4. I tried using devtools (as outlined in the repo) to install the latest version from GitHub, but I run into this error:

   Error: processing vignette 'FlowSOM.Rnw' failed with diagnostics:
   Running 'texi2dvi' on 'FlowSOM.tex' failed.
   LaTeX errors:
   ! Undefined control sequence.
   l.320 \code
              {PlotFlowSOM} or you can use the \code{PlotStars} function.
   The control sequence at the end of the top line
   of your error message was never \def'ed. If you have
   ! Undefined control sequence.
   l.320 \code{PlotFlowSOM} or you can use the \code
                                                    {PlotStars} function.
   The control sequence at the end of the top line
   of your error message was never \def'ed. If you have
   ! Undefined control sequence.
   l.345 node, you can use the parameter \code
                                              {equalNodeSize = T} in every plot...
   The control sequence at the end of the top line
   of your error message was never \def'ed. If you have
   --- failed re-building ‘FlowSOM.Rnw’

Is there an easy fix to this, and if not, how can I download version 1.4.0 as outlined in the RDocumentation page so I can access MapDataToCodes?

Long Vectors error in SOM Function (Analogous to issue #54)

I previously noted an issue that arose in the MapDataToCodes step in which there is a long vectors error because the size of the data is larger than 2^31-1, which creates an error in the referenced C code: #54

The solution on this worked great. Now I am increasing the size of the data even further and encountering an analogous error in the SOM function.

somResults <- SOM(data=as.matrix(pixelData), rlen=numPasses, xdim=10, ydim=10, alpha=c(lr_start, lr_end)

And the error is long vectors are not supported in .C

Since you had a written solution to the previous issue, I am wondering, do you have a solution in this case by breaking down the object into blocks, or is that not possible in this case?

Thanks,
Jonathan

Transposed the metaclusterMFI result!!

Hi. Thanks for updating the package. I have just read this major change. Although it only targets the meta clustering, I wonder how much it will impact packages (such as cytofkit) and analyses that depend on it.
Best.

SOM function: check codes vs grid and data

Hi,
May I suggest that, if codes is not null, the SOM function should verify:

• ncol(codes) == ncol(data)
• nrow(codes) == xdim * ydim

FlowSOM/R/2_buildSOM.R

Lines 95 to 98 in af08bb6

	SOM <- function (data, xdim=10, ydim=10, rlen=10, mst=1, alpha=c(0.05, 0.01),
	radius = stats::quantile(nhbrdist, 0.67) * c(1, 0),
	init=FALSE, distf=2, silent=FALSE, codes=NULL,
	importance = NULL)

Cheers.

Change scale settings on MST

Using the FlowSOM plugin on FlowJo, I've generated MSTs for two sample types (both colored on a single marker of interest).

Both samples express the marker at different levels, but the MSTs are scaled and colored individually (one from ~200-550; the other from ~200-650). Accordingly, it is impossible to tell the difference between the samples by comparing color intensity alone (i.e., the min/max intensities are different, but they are both colored the same shades of blue/red).

Does anyone know if (and how) it is possible to modify the scale settings so they are the same (e.g., ~200-650) on both MSTs?

Any guidance would be gratefully accepted.

Best,

Ben

NewData: doc & example

Hi. The doc states that

We assume the data is already compensated and transformed, but not scaled yet. The same scaling parameters as from the original grid will be used.

The code shows that if compensation/transform/scale slots are present in the flowSOM object those operations are applied to the flowframe.

FlowSOM/R/3_buildMST.R

Lines 1794 to 1811 in 50daac2

	if(fsom$compensate){
	ff <- flowCore::compensate(ff, fsom$spillover)
	}
	if(fsom$transform){
	ff <- flowCore::transform(ff, flowCore::transformList(
	BiocGenerics::colnames(ff[, fsom$toTransform]),
	fsom$transformFunction))
	}
	if(fsom$scale){
	newData <- scale(exprs(ff),
	center = fsom$scaled.center[
	BiocGenerics::colnames(ff)],
	scale = fsom$scaled.scale[
	BiocGenerics::colnames(ff)])
	} else {
	newData <- exprs(ff)
	}

I think the doc should be updated accordingly. The example should also be changed because a flowframe might be compensated and transformed twice currently leading to wrong interpretations.
Let me know if I should setup a PR. Best.

SaveClustersToFCS

Hello,

I have used SaveClustersToFCS and imported the newly created .fcs files in R or flowjo. When I then check the node information added in the FlowSOM column with either of the softwares I would expect that each cell had a value between 1 and 100 as I had 100 nodes in my FlowSOM run.

However, this are the values that I am obtaining:

unique(exprs(fcs[[1]]$FlowSOM))
FlowSOM
[1,] 10
[2,] 8
[3,] 96
[4,] 47
[5,] 3
[6,] 65
[7,] 2
[8,] 9

Could you help me understand where these values come from? Also, I would rather link the information of the metaclusters than the nodes. Is this possible?

Thank you very much for your help!

Can FlowSOM be used for scRNA-seq data?

Hello everyone,
I use FlowSOM in cytobank for CyTOF data, I was wondering if anyone have applied FlowSOM to scRNA-seq and compared it with other clustering methods?

Thank you!

No output

hello I'm seeing the following message after inputing my data/controls. Can anyone assist?

Error in FlowSOM::AggregateFlowFrames(files, nCells, channels = colsToUse) :
unused argument (channels = colsToUse)

Loading FlowSOM package fails

Hi,

when I try to load the FlowSOM package, then I receive an error message:

> library(FlowSOM)
Error: package or namespace load failed for ‘FlowSOM’: object ‘compute_timestep’ is not exported by 'namespace:flowWorkspace'

I am running macOS Catalina 10.15.7 and R version 4.0.3.
FlowSOM was installed via "BiocManager::install("FlowSOM")" and the installed FlowSOM version is 1.20.0
I tried to install FlowSOM via devtools::install_github("SofieVG/FlowSOM") or devtools::install_github("SofieVG/FlowSOM", ref = "devel") in case there is a more recent version. But the FlowSOM installation from github fails in both cases (see the error messages copied below).

I tried to load FlowSOM in combination with two different flowWorkspace versions: Version 4.0.6 installed via BiocManager and Version 4.1.12 installed from github. The flowWorkspace version does not have an effect on the error while loading FlowSOM.

Can you please advice how to fix loading FlowSOM?
Thank you very much in advance!

Error messages during the installation of FlowSOM from github:

devtools::install_github("SofieVG/FlowSOM")
Downloading GitHub repo SofieVG/FlowSOM@HEAD
✓ checking for file ‘/private/var/folders/sl/6h6dh39d2p3617w8h3sf2bsm0000gn/T/RtmpR2sVJa/remotes10cf62afa7f5e/SofieVG-FlowSOM-6f79b8b/DESCRIPTION’ ...
─ preparing ‘FlowSOM’:
✓ checking DESCRIPTION meta-information ...
─ cleaning src
─ checking for LF line-endings in source and make files and shell scripts
─ checking for empty or unneeded directories
─ building ‘FlowSOM_1.19.4.tar.gz’

• installing source package ‘FlowSOM’ ...
  ** using staged installation
  ** libs
  clang -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/usr/local/include -fPIC -Wall -g -O2 -c som.c -o som.o
  som.c:37:18: warning: using integer absolute value function 'abs' when argument is of floating point type [-Wabsolute-value]
  xdist += abs(tmp);
  ^
  som.c:37:18: note: use function 'fabs' instead
  xdist += abs(tmp);
  ^~~
  fabs
  som.c:47:15: warning: using integer absolute value function 'abs' when argument is of floating point type [-Wabsolute-value]
  tmp = abs(tmp);
  ^
  som.c:47:15: note: use function 'fabs' instead
  tmp = abs(tmp);
  ^~~
  fabs
  som.c:134:27: warning: using integer absolute value function 'abs' when argument is of floating point type [-Wabsolute-value]
  change += abs(tmp);
  ^
  som.c:134:27: note: use function 'fabs' instead
  change += abs(tmp);
  ^~~
  fabs
  3 warnings generated.
  clang -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -o FlowSOM.so som.o -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
  installing to /Library/Frameworks/R.framework/Versions/4.0/Resources/library/00LOCK-FlowSOM/00new/FlowSOM/libs
  ** R
  ** inst
  ** byte-compile and prepare package for lazy loading
  Error in dyn.load(file, DLLpath = DLLpath, ...) :
  unable to load shared object '/Library/Frameworks/R.framework/Versions/4.0/Resources/library/flowWorkspace/libs/flowWorkspace.so':
  dlopen(/Library/Frameworks/R.framework/Versions/4.0/Resources/library/flowWorkspace/libs/flowWorkspace.so, 6): Symbol not found: __ZN7cytolib11H5CytoFrameC1ERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEERNS_14FCS_READ_PARAMES9_b
  Referenced from: /Library/Frameworks/R.framework/Versions/4.0/Resources/library/flowWorkspace/libs/flowWorkspace.so
  Expected in: flat namespace
  in /Library/Frameworks/R.framework/Versions/4.0/Resources/library/flowWorkspace/libs/flowWorkspace.so
  Calls: ... asNamespace -> loadNamespace -> library.dynam -> dyn.load
  Execution halted
  ERROR: lazy loading failed for package ‘FlowSOM’
• removing ‘/Library/Frameworks/R.framework/Versions/4.0/Resources/library/FlowSOM’
• restoring previous ‘/Library/Frameworks/R.framework/Versions/4.0/Resources/library/FlowSOM’
  Error: Failed to install 'FlowSOM' from GitHub:
  (converted from warning) installation of package ‘/var/folders/sl/6h6dh39d2p3617w8h3sf2bsm0000gn/T//RtmpR2sVJa/file10cf623c00a5d/FlowSOM_1.19.4.tar.gz’ had non-zero exit status

Error while using CountGroups Function

groupRes<-CountGroups(fSOM[[1]], fgroups)

Getting the following error while running the command above:

[1]"/path/Sample.fcs"
Error in newdata[, colnames(codes)] : subscript out of bounds

importing the gating workspace from flowJo as .xml file in order to work in flowSOM

Hi,
I've been exploring FlowSOM and had a problem when exporting .xml. I've tried several ways:

• get the XML code from flowJo & saving it through a text document as .xml file
• importing it as .wsp file and opening it through R
  However, each time I get the following error:
  Error in parseGatingML(xmlRoot(smartTreeParse(file, ...)), flowEnv) :
  Not a supported Gating-ML XML format
  I would appreciate any help.
  Thank you

error while parsing workspace with fcs files

Hello,

I've been trying to parse the workspace from FlowJo with my fcs files using GetFlowJoLabels function. However, I get this error message:

Parsing 0 samples
windows version of flowJo workspace recognized.
version X
Error in flowWorkspace:::.addGatingHierarchies(gs, pd, execute, isNcdf, :
no sample to be added to GatingSet!
In addition: Warning message:
In .parse.pData(obj = obj, keywords = keywords, sg = sg, keywords.source = keywords.source, :
Can't find the FCS files for the following samples:
9 days after sorting_60220_coGFP_002.fcs_10000

I am sure that I have provided the right path to my files and the workspace.
For some reason my workspace adds to the name of the file "_1000" extension (Original file name is 9 days after sorting_60220_coGFP_002.fcs) .
I have tried to parse the same ws and files using OpenCyto and it worked. As soon as I try with GetFlowJoLabes function it fails.

Could you please tell me where it goes wrong?

Kind regards

Add option to return just the weights from the`SOM` function

Calling the SOM function will currently generate the weights of the SOM and internally run MapDataToCodes on the dataset. However, the user may not want to immediately run MapDataToCodes on the dataset. For example, to save memory, the user may decide to train the SOM on just a subset of the data, then use the weights to directly invoke MapDataToCodes on the entire dataset.

The SOM function should include a flag to allow the user to disable the call to MapDataToCodes and just save the SOM weights for applications such as this.

Elbow plot variance calculation

Hi, I want to ask how the following formula for calculating the variance (that will be used in the elbow plot) is derived?

  c_wss <- 0
  for(j in seq_along(clustering)){
    if(sum(clustering == j) > 1){
      c_wss <- c_wss + (nrow(data[clustering == j, , drop = FALSE]) - 1) *
        sum(apply(data[clustering == j, , drop = FALSE], 2, stats::var))
    }
  }

I understand that the sum() part is calculating the within sum of squares but why does it have to be multiplied by what I assume is the degrees of freedom with the nrow() - 1? Thanks a lot!

AggregateFlowFrames with channels given produces error

I get an "Error: Subset out of bounds" error when using AggreateFlowFrames.

I got a reproducible example (see below) after debugging the cytoNorm examples...

> library(FlowSOM)
> fileName <- system.file("extdata", "68983.fcs", package = "FlowSOM")
> f=flowCore::read.FCS(fileName)
> channels = unname(flowCore::parameters(f)@data[["name"]])[c(2,5,8,9,10)]
> ff_new <- AggregateFlowFrames(c(fileName, fileName), 1000,channels = channels)
Reading /Library/Frameworks/R.framework/Versions/4.2/Resources/library/FlowSOM/extdata/68983.fcs
Error: Subset out of bounds
> sessionInfo()
R version 4.2.1 (2022-06-23)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Monterey 12.6

Matrix products: default
LAPACK: /Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] FlowSOM_2.5.8 igraph_1.3.5 

loaded via a namespace (and not attached):
 [1] Biobase_2.56.0              httr_1.4.4                  tidyr_1.2.1                 jsonlite_1.8.1             
 [5] ConsensusClusterPlus_1.60.0 carData_3.0-5               RcppParallel_5.1.5          assertthat_0.2.1           
 [9] stats4_4.2.1                latticeExtra_0.6-30         RBGL_1.72.0                 flowWorkspace_4.9.1        
[13] yaml_2.3.5                  pillar_1.8.1                backports_1.4.1             lattice_0.20-45            
[17] glue_1.6.2                  digest_0.6.29               RColorBrewer_1.1-3          ggsignif_0.6.3             
[21] polyclip_1.10-0             colorspace_2.0-3            ggcyto_1.25.1               plyr_1.8.7                 
[25] ggnewscale_0.4.7            XML_3.99-0.10               pkgconfig_2.0.3             broom_1.0.1                
[29] zlibbioc_1.42.0             purrr_0.3.4                 flowCore_2.9.1              scales_1.2.1               
[33] tweenr_2.0.2                jpeg_0.1-9                  Rtsne_0.16                  ggforce_0.4.0              
[37] tibble_3.1.8                aws.s3_0.3.21               farver_2.1.1                generics_0.1.3             
[41] car_3.1-0                   ggplot2_3.3.6               ggpubr_0.4.0                withr_2.5.0                
[45] BiocGenerics_0.42.0         hexbin_1.28.2               cli_3.4.1                   magrittr_2.0.3             
[49] deldir_1.0-6                fansi_1.0.3                 MASS_7.3-58.1               rstatix_0.7.0              
[53] xml2_1.3.3                  graph_1.74.0                tools_4.2.1                 data.table_1.14.2          
[57] ncdfFlow_2.43.1             lifecycle_1.0.2             matrixStats_0.62.0          interp_1.1-3               
[61] S4Vectors_0.34.0            munsell_0.5.0               cluster_2.1.4               colorRamps_2.3.1           
[65] compiler_4.2.1              rlang_1.0.6                 grid_4.2.1                  rstudioapi_0.14            
[69] aws.signature_0.6.0         base64enc_0.1-3             cytolib_2.9.1               gtable_0.3.1               
[73] abind_1.4-5                 DBI_1.1.3                   curl_4.3.2                  R6_2.5.1                   
[77] RProtoBufLib_2.8.0          gridExtra_2.3               dplyr_1.0.10                utf8_1.2.2                 
[81] CytoML_2.8.1                Rgraphviz_2.40.0            Rcpp_1.0.9                  vctrs_0.4.2                
[85] png_0.1-7                   tidyselect_1.1.2           
> 

File does not exist

Hi there,

when we apply flowSOM in FlowJo to one of our populations the flowSOM csv output file is missing. Instead the only file we can find in the flowSOM folder is the following:
SYN 101_Tube_004_015.LiveCD3+.ExtNode.csv

Please see the below FlowJo engine log:

<FCML engineId="ExE974859553" > <FcmlQuery msg="File does not exist: D:\xxxxx\FlowSOM\SYN_101_Tube_004_015.LiveCD3+.csv.flowSOM.csv" > <FlowSOM version="1.0" showOutput="1" timeInMs="32" > <Option view="MST" /> <Option pdf="1" /> <Option fixedNodeSize="1" /> <Option MFIsInTable="0" /> <Option plot="All as piecharts" /> <Option applyOn="None" /> <Option ydim="10" /> <Option xdim="10" /> <Option scale="1" /> <Option nodeScale="100.0" /> <Option nClus="8" /> <Option bgMetaClusters="1" /> <Parameter name="Comp-FITC-A" /> <Parameter name="Comp-PE-A" /> <Parameter name="Comp-PerCP-Cy5-5-A" /> <Parameter name="Comp-APC-A" /> <Parameter name="Comp-BV 421-A" /> </FlowSOM> <Export/> <Error/> </FcmlQuery> </FCML>

SaveClusters ToFCS with multiple data segments

In R console I call FlowSOM() to load an FCS and apply clustering and metaclustering and then call SaveClustersToFCS() to save the modified file.

However, SaveClustersToFCS reports an error to me (see below), it looks like FlowSOM can't load or save FCS files with multiple data segments? Is that correct?

Thanks

> fileName <- "MyFile.LMD"
> cols <- c(3:7)

> fsom <- FlowSOM(fileName, pattern=".LMD", compensate=FALSE,transform=FALSE,scale=FALSE,colsToUse=cols, nClus=20)
> SaveClustersToFCS(fsom,fileName)
Extending MyFile.LMD using the FlowSOMmapping of MyFile.LMD indexed by 
Error in m[s, ] <- fsom_f$map$mapping[, 1] : 
  number of items to replace is not a multiple of replacement length
In addition: Warning messages:
1: The file contains 1 additional data segment.
The default is to read the first segment only.
Please consider setting the 'dataset' argument. 
2: Invalid bitwidth specification.
This is a known bug in Beckman Coulter's CPX software.
The data might be corrupted if produced by another software. 
3: The file contains 1 additional data segment.
The default is to read the first segment only.
Please consider setting the 'dataset' argument. 
4: Invalid bitwidth specification.
This is a known bug in Beckman Coulter's CPX software.
The data might be corrupted if produced by another software. 
5: In NewData(fsom, ff) :
  5469 cells (1.63%) seem far from their cluster centers.

getIndiceMat not exported by new flowWorkspace version

Thanks for a great package!

Just a heads up, looks like getIndiceMat is no longer exported by the new version of flowWorkspace which is causing build errors. This is using the current GitHub version from the trunk branch 3.33.10.

Error: object 'getIndiceMat' is not exported by 'namespace:flowWorkspace'
Execution halted
ERROR: lazy loading failed for package 'FlowSOM'

Cheers,
Dillon

All populations not exported into FlowJo

Hi, I am having an issue when using the FlowJo plugin where I select 40 metaclusters, but am only getting 20 populations (0-19) exported into FlowJo as opposed to a number closer to 40. Everything else runs fine, its just the population export -- this might be a FlowJo issue, but I am not sure. Thanks!